Skip to content

Add column range filtering and ignore columns in comparison - #3565

Open
sdottaka wants to merge 1 commit into
masterfrom
feature/ignore-column-in-comparison
Open

Add column range filtering and ignore columns in comparison#3565
sdottaka wants to merge 1 commit into
masterfrom
feature/ignore-column-in-comparison

Conversation

@sdottaka

Copy link
Copy Markdown
Member

Summary

  • Add the ColumnRange() filter expression for selecting or excluding multiple table columns.
  • Add a context menu command to ignore a column in all panes or only the current pane during comparison.
  • Support adding multiple ignored columns and resetting ignored column settings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds ColumnRange() filtering and context-menu controls for ignoring columns during comparisons.

Changes:

  • Adds column-range selection/exclusion, tests, and documentation.
  • Adds ignore-column commands for all panes, the current pane, multiple columns, and reset.
  • Updates providers, document/view integration, resources, and file transformations.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.

Show a summary per file
File Summary Final review comments
Testing/GoogleTest/FilterEngine/FilterExpression_test.cpp Adds range-filter tests and provider support. No final comments.
Src/resource.h Defines new command IDs. No final comments.
Src/MergeEditView.h Declares ignore-column handlers. No final comments.
Src/MergeEditView.cpp Handles ignore-column commands and UI state. Moderate (2 votes): The command remains enabled when plugins are disabled, so it updates the pipeline without changing comparison results.
Src/MergeDoc.h Adds ignore-column and range APIs. No final comments.
Src/MergeDoc.cpp Updates ignored-column pipelines and column extraction. Moderate (2 votes): User-authored bare ColumnRange(...) expressions can be rewritten as generated ignore state, changing selection into its complement.
Src/Merge.rc Adds the ignore-column menu. Critical (3 votes): The MENUITEM is missing the required comma and may prevent the resource from compiling.
Src/FilterEngine/ILineDataProvider.h Extends the provider interface. No final comments.
Src/FilterEngine/FilterExpressionNodes.h Declares ColumnRange support. No final comments.
Src/FilterEngine/FilterExpressionNodes.cpp Parses and evaluates column ranges. Moderate (3 votes): Large user-supplied range endpoints can cause billions of unnecessary iterations and overflow; stop once the column count is reached.
Src/FileTransform.cpp Integrates range extraction into transformations. Critical (1 vote): A right-pane unpacking path can index filenames out of bounds. Critical (1 vote): Non-table files can receive NUL delimiters when selecting multiple columns, corrupting text output.
Docs/Manual/English/Filters.xml Documents ColumnRange() syntax and behavior. No final comments.
Suppressed comments (4)

Src/FileTransform.cpp:949

  • The prediff provider re-detects table properties from the target filename, but the editor's table mode can use shared properties from another pane or a user-selected delimiter (CMergeDoc::SetTableProperties). In a CSV-vs-TXT comparison, or after “Recompare as Table” with a custom delimiter, this target can be treated as one column with delimiter 0; ColumnRange("!1") then rewrites every line to empty instead of removing the selected field. Pass the effective comparison table properties into the provider rather than deriving them only from the filename.
	const auto filenames = strutils::split(filteredFilenames, '|');
	auto tableProps = MakeTablePropertiesByFileName(String(filenames[target].data(), filenames[target].size()));

Src/FilterEngine/FilterExpressionNodes.cpp:4026

  • The included-range loop has the same unbounded-iteration problem: ColumnRange("1-2147483647") scans up to two billion values even when the line has only a few columns, and may overflow at INT_MAX. Bound the loop by columnCount before incrementing.
			for (int column = range.start; column <= end; ++column)

Src/FilterEngine/FilterExpressionNodes.cpp:3915

  • Malformed range specifications throw std::invalid_argument, but the parser maps every invalid_argument from optimization to FILTER_ERROR_DIVIDE_BY_ZERO (Src/FilterEngine/FilterParser.y:59-63). An invalid value such as ColumnRange("") is consequently reported to users as a division-by-zero error instead of an invalid column specification. Use a dedicated error category or preserve the actual validation error.
		throw std::invalid_argument("invalid column specification");

Src/MergeDoc.cpp:363

  • This reset removes every pure ColumnRange("...") line expression, including expressions a user added manually through the documented ColumnRange feature for a prediffer. Therefore Reset All can silently delete an unrelated custom column-selection filter. Track expressions created by this command or otherwise distinguish ignored-column state from user-authored range filters before removing them.
		if (lineExpression &&
			lineExpression->expression.find(_T("ColumnRange(\"")) == 0 &&
			lineExpression->expression.size() >= String(_T("ColumnRange(\"")).size() + 2 &&
			lineExpression->expression.compare(lineExpression->expression.size() - 2, 2, _T("\")")) == 0)
			continue;

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Src/FileTransform.cpp
Comment on lines +948 to +949
const auto filenames = strutils::split(filteredFilenames, '|');
auto tableProps = MakeTablePropertiesByFileName(String(filenames[target].data(), filenames[target].size()));
Comment thread Src/FileTransform.cpp
Comment on lines +842 to +843
const std::string delimiter = ucr::toUTF8(String(&m_tableProps.delimiter, 1));
std::string result;
if (!range.excluded)
continue;
const int end = range.end == -1 ? columnCount : range.end;
for (int column = range.start; column <= end; ++column)
Comment thread Src/Merge.rc
BEGIN
MENUITEM "In &All Panes", ID_IGNORE_COLUMN_IN_COMPARISON
MENUITEM "In &This Pane Only (Column Not in Other Files)", ID_IGNORE_COLUMN_IN_COMPARISON_THIS_PANE_ONLY
MENUITEM "(Ctrl+Click to add another column)" ID_IGNORE_COLUMN_IN_COMPARISON_CTRL_CLICK_TO_ADD
Comment thread Src/MergeDoc.cpp
Comment on lines +306 to +310
if (!lineExpression || lineExpression->targetFlags != targetFlags ||
lineExpression->expression.compare(0, expressionPrefix.size(), expressionPrefix) != 0 ||
lineExpression->expression.size() < expressionPrefix.size() + columnSuffix.size() ||
lineExpression->expression.compare(lineExpression->expression.size() - columnSuffix.size(),
columnSuffix.size(), columnSuffix) != 0)
Comment thread Src/MergeEditView.cpp
Comment on lines +4469 to +4470
pCmdUI->Enable(m_nClickedColumn >= 0 &&
GetDocument()->m_ptBuf[m_nThisPane]->GetTableEditing());
bool excluded;
};

static int ParseColumnNumber(const std::string& text, size_t& pos)
const int end = range.end == -1 ? columnCount : range.end;
for (int column = range.start; column <= end; ++column)
{
const size_t index = static_cast<size_t>(column - 1);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants