fix(HF-135): enforce array dimensions and spill boundaries

CHANGELOG.md

@@ -9,6 +9,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ### Fixed
 
+- Fixed array results exceeding the configured row or column limits, including intermediate arrays created by `VSTACK`, `HSTACK`, `TRANSPOSE`, and array literals. Arrays with valid dimensions now return `#SPILL!` when their spill area extends beyond the sheet edge. [#1774](https://github.com/handsontable/hyperformula/pull/1774)
 - Fixed the `AVERAGEIF` function returning a division-by-zero error when the calculated average was `0`. [#1733](https://github.com/handsontable/hyperformula/pull/1733)
 - Fixed the localized names of `VSTACK` and `HSTACK` in 14 language packs to match Microsoft Excel. [#1748](https://github.com/handsontable/hyperformula/pull/1748)
 - Fixed the MAXPOOL and MEDIANPOOL functions throwing an uncaught `TypeError` instead of returning the `#VALUE!` error when the range dimensions are not a whole multiple of the window size and the stride. [#1718](https://github.com/handsontable/hyperformula/pull/1718)

docs/guide/arrays.md

@@ -157,6 +157,16 @@ If your specified output array size is larger or equal to the input array size,
 
 ## Array rules
 
+### Array size limits
+
+HyperFormula limits array dimensions using the [`maxRows`](../api/interfaces/configparams.md#maxrows) and [`maxColumns`](../api/interfaces/configparams.md#maxcolumns) configuration options. An array result that exceeds either limit returns `#VALUE!` (`Value too large.`). This also applies to intermediate arrays: wrapping an oversized `VSTACK` result in `SUM` or `ARRAY_CONSTRAIN` does not bypass the limits. Reduce the dimensions of the intermediate array to resolve the error.
+
+An array whose dimensions fit within these limits can still extend beyond the sheet edge because of its formula's position. In that case, HyperFormula returns `#SPILL!` (`No space for array result.`). Move the formula to a position with enough space or reduce its output dimensions. For example, with `maxRows` set to `4`, `=SEQUENCE(4)` fits at `A1` but returns `#SPILL!` at `A2`. A scalar formula such as `=SUM(SEQUENCE(4))` can be placed at `A2` because its result occupies only one cell.
+
+An array formula that returns `#SPILL!` does not spill again on its own when space becomes available, for example after the rows above it are removed or a blocking value is cleared. Re-enter the formula to recalculate it. See [#1785](https://github.com/handsontable/hyperformula/issues/1785).
+
+These options limit each dimension separately. They do not impose a separate limit on the total number of cells in an array, so an array within both limits can still be very large: at the default limits, `=SEQUENCE(40000, 18278)` has over 731 million cells and can exhaust the available memory.
+
 ### With the array arithmetic mode enabled
 
 When the [array arithmetic mode](#array-arithmetic-mode) is enabled, and you pass an array to a [scalar](#about-arrays) function, the following rules apply:

docs/guide/list-of-differences.md

@@ -52,6 +52,7 @@ See a full list of differences between HyperFormula, Microsoft Excel, and Google
 | EDATE function                                     | =EDATE(DATE(2019, 7, 31), 1)                                              | Type of the returned value: `CellValueDetailedType.NUMBER_DATE`. This is non-compliant with the [OpenDocument](https://docs.oasis-open.org/office/OpenDocument/v1.3/os/part4-formula/OpenDocument-v1.3-os-part4-formula.html) standard, which defines the return type as a Number, while describing it as a Date serial number through the function summary. | Cell auto-formatted as **date**                                                                                                | Cell auto-formatted as **regular number**                                                                                      |
 | EOMONTH function                                   | =EOMONTH(DATE(2019, 7, 31), 1)                                            | Type of the returned value: `CellValueDetailedType.NUMBER_DATE`. This is non-compliant with the [OpenDocument](https://docs.oasis-open.org/office/OpenDocument/v1.3/os/part4-formula/OpenDocument-v1.3-os-part4-formula.html) standard, which defines the return type as a Number, while describing it as a Date serial number through the function summary. | Cell auto-formatted as **date**                                                                                                | Cell auto-formatted as **regular number**                                                                                      |
 | Empty cells in lookup search                      | =XLOOKUP(3, A1:A4, A1:A4, "NF", 0, 2)<br>where A1:A4 = 1, 2, (empty), 3   | Empty cells are skipped during the search. A value that is present is found even when it sits past an interspersed empty cell (exact match is gap-independent), and approximate `MATCH`/`VLOOKUP`/`HLOOKUP`/`XLOOKUP` skip empty cells — but not empty strings — when finding the lower/upper bound. Returns `3`. On an all-empty range in a binary search mode, HyperFormula returns the `if_not_found` result (never row 1). | Skips empty cells in approximate search (parity with HyperFormula). | With binary search modes (`search_mode` ±2), a range with interspersed empty cells is not strictly sorted; per Excel's documentation the result may be invalid, so a value past an empty cell is not reliably found. On an all-empty range in a binary mode, Excel returns the first row's value. |
+| Array size limits                                  | =SUM(VSTACK(SEQUENCE(1048576), SEQUENCE(1)))<br><br>=SUM(HSTACK(SEQUENCE(1, 18278), SEQUENCE(1, 1))) | Every array, including an intermediate array that is never spilled, is limited in both dimensions by the [`maxRows`](../api/interfaces/configparams.md#maxrows) and [`maxColumns`](../api/interfaces/configparams.md#maxcolumns) configuration options. An array that exceeds either limit returns `#VALUE!`. With the default limits (40,000 rows, 18,278 columns), both examples return `#VALUE!`. | In-memory arrays are not limited in either dimension; only a spilled result is limited by the sheet size. Returns `549756338177` and `167051782`. | In-memory arrays are limited to 1,048,576 rows, but not in columns; an over-wide array fails only when it spills, with `#SPILL!`. Returns `#NUM!` for the first example. |
 
 ## Built-in functions
 

src/ArraySize.ts

@@ -29,6 +29,14 @@ export class ArraySize {
   isScalar(): boolean {
     return (this.width === 1 && this.height === 1) || this.isRef
   }
+
+  /**
+   * Checks result dimensions against the configured sheet limits, independently
+   * of the formula's position. Call before allocating a result array.
+   */
+  exceedsSheetSizeLimits(maxColumns: number, maxRows: number): boolean {
+    return this.width > maxColumns || this.height > maxRows
+  }
 }
 
 function arraySizeForBinaryOp(leftArraySize: ArraySize, rightArraySize: ArraySize): ArraySize {
@@ -46,8 +54,18 @@ export class ArraySizePredictor {
   ) {
   }
 
+  /**
+   * Limits the predicted spill area before the dependency graph maps its cells.
+   * A scalar fallback still evaluates the formula, so runtime allocation guards
+   * determine the error and also protect intermediate arrays.
+   * Infinite dimensions represent whole-row or whole-column references, whose
+   * effective size is resolved by the dependency graph.
+   */
   public checkArraySize(ast: Ast, formulaAddress: SimpleCellAddress): ArraySize {
-    return this.checkArraySizeForAst(ast, {formulaAddress, arraysFlag: this.config.useArrayArithmetic})
+    const size = this.checkArraySizeForAst(ast, {formulaAddress, arraysFlag: this.config.useArrayArithmetic})
+    const maxColumns = size.width === Infinity ? Infinity : this.config.maxColumns
+    const maxRows = size.height === Infinity ? Infinity : this.config.maxRows
+    return size.exceedsSheetSizeLimits(maxColumns, maxRows) ? ArraySize.error() : size
   }
 
   public checkArraySizeForAst(ast: Ast, state: InterpreterState): ArraySize {

src/DependencyGraph/DependencyGraph.ts

@@ -63,6 +63,7 @@ export class DependencyGraph {
     public readonly lazilyTransformingAstService: LazilyTransformingAstService,
     public readonly functionRegistry: FunctionRegistry,
     public readonly namedExpressions: NamedExpressions,
+    private readonly config: Config,
   ) {
     this.graph = new Graph<Vertex>(this.dependencyQueryVertices)
     this.sheetReferenceRegistrar = new SheetReferenceRegistrar(sheetMapping, addressMapping)
@@ -82,7 +83,8 @@ export class DependencyGraph {
       stats,
       lazilyTransformingAstService,
       functionRegistry,
-      namedExpressions
+      namedExpressions,
+      config
     )
   }
 
@@ -468,7 +470,7 @@ export class DependencyGraph {
 
   public isThereSpaceForArray(arrayVertex: ArrayFormulaVertex): boolean {
     const range = arrayVertex.getRangeOrUndef()
-    if (range === undefined) {
+    if (range === undefined || range.exceedsSheetSizeLimits(this.config.maxColumns, this.config.maxRows)) {
       return false
     }
     for (const address of range.addresses(this)) {

src/interpreter/Interpreter.ts

@@ -4,7 +4,7 @@
  */
 
 import {AbsoluteCellRange, AbsoluteColumnRange, AbsoluteRowRange} from '../AbsoluteCellRange'
-import {ArraySizePredictor} from '../ArraySize'
+import {ArraySize, ArraySizePredictor} from '../ArraySize'
 import {ArrayValue, NotComputedArray} from '../ArrayValue'
 import {CellError, ErrorType, isColOrRowInvalid} from '../Cell'
 import {Config} from '../Config'
@@ -256,15 +256,20 @@ export class Interpreter {
           for (const astIt of astRow) {
             const arr = coerceToRange(this.evaluateAst(astIt, state))
             const height = arr.height()
+            if (rowHeight !== undefined && rowHeight !== height) {
+              return new CellError(ErrorType.REF, ErrorMessage.SizeMismatch)
+            }
+            const size = new ArraySize((rowRet[0]?.length ?? 0) + arr.width(), ret.length + height)
+            if (size.exceedsSheetSizeLimits(this.config.maxColumns, this.config.maxRows)) {
+              return new CellError(ErrorType.VALUE, ErrorMessage.ValueLarge)
+            }
             if (rowHeight === undefined) {
               rowHeight = height
               rowRet.push(...arr.data)
-            } else if (rowHeight === height) {
+            } else {
               for (let i = 0; i < height; i++) {
                 rowRet[i].push(...arr.data[i])
               }
-            } else {
-              return new CellError(ErrorType.REF, ErrorMessage.SizeMismatch)
             }
           }
           const width = rowRet[0].length

src/interpreter/plugin/ArrayPlugin.ts

@@ -180,6 +180,10 @@ export class ArrayPlugin extends FunctionPlugin implements FunctionPluginTypeche
   public vstack(ast: ProcedureAst, state: InterpreterState): InterpreterValue {
     return this.runFunction(ast.args, state, this.metadata('VSTACK'), (...ranges: SimpleRangeValue[]) => {
       const width = Math.max(...ranges.map(range => range.width()))
+      const height = ranges.reduce((total, range) => total + range.height(), 0)
+      if (new ArraySize(width, height).exceedsSheetSizeLimits(this.config.maxColumns, this.config.maxRows)) {
+        return new CellError(ErrorType.VALUE, ErrorMessage.ValueLarge)
+      }
       const result: InternalScalarValue[][] = []
 
       for (const range of ranges) {
@@ -224,6 +228,10 @@ export class ArrayPlugin extends FunctionPlugin implements FunctionPluginTypeche
   public hstack(ast: ProcedureAst, state: InterpreterState): InterpreterValue {
     return this.runFunction(ast.args, state, this.metadata('HSTACK'), (...ranges: SimpleRangeValue[]) => {
       const height = Math.max(...ranges.map(range => range.height()))
+      const width = ranges.reduce((total, range) => total + range.width(), 0)
+      if (new ArraySize(width, height).exceedsSheetSizeLimits(this.config.maxColumns, this.config.maxRows)) {
+        return new CellError(ErrorType.VALUE, ErrorMessage.ValueLarge)
+      }
       const result: InternalScalarValue[][] = [...Array(height).keys()].map(() => [])
 
       for (const range of ranges) {

src/interpreter/plugin/MatrixPlugin.ts

@@ -298,8 +298,11 @@ export class MatrixPlugin extends FunctionPlugin implements FunctionPluginTypech
 
   public transpose(ast: ProcedureAst, state: InterpreterState): InterpreterValue {
     return this.runFunction(ast.args, state, this.metadata('TRANSPOSE'), (matrix: SimpleRangeValue) => {
-      const input = matrix.rawData()
       const inputSize = matrix.size
+      if (new ArraySize(inputSize.height, inputSize.width).exceedsSheetSizeLimits(this.config.maxColumns, this.config.maxRows)) {
+        return new CellError(ErrorType.VALUE, ErrorMessage.ValueLarge)
+      }
+      const input = matrix.rawData()
       const result: InternalScalarValue[][] = []
       for (let i = 0; i < inputSize.width; ++i) {
         result[i] = []

src/interpreter/plugin/SequencePlugin.ts

@@ -108,7 +108,7 @@ export class SequencePlugin extends FunctionPlugin implements FunctionPluginType
           return new CellError(ErrorType.VALUE, ErrorMessage.LessThanOne)
         }
 
-        if (numRows > this.config.maxRows || numCols > this.config.maxColumns) {
+        if (new ArraySize(numCols, numRows).exceedsSheetSizeLimits(this.config.maxColumns, this.config.maxRows)) {
           return new CellError(ErrorType.VALUE, ErrorMessage.ValueLarge)
         }
 
@@ -167,10 +167,11 @@ export class SequencePlugin extends FunctionPlugin implements FunctionPluginType
       return ArraySize.error()
     }
 
-    if (rows > this.config.maxRows || cols > this.config.maxColumns) {
+    const size = new ArraySize(cols, rows)
+    if (size.exceedsSheetSizeLimits(this.config.maxColumns, this.config.maxRows)) {
       return ArraySize.error()
     }
 
-    return new ArraySize(cols, rows)
+    return size
   }
 }