Resolves: tdf#98389 check memory requirement of matrix

Attachment 123203 of tdf#98389 has external references of the form
[1]Data!$1:$1048576
which effectively address the entire cell range of sheet Data and
allocating a matrix for a billion cells attempted to allocate 8GB+xGB of
memory which the system said no-no ...

Some brave soul along the road once decided that a check on the number
of elements wasn't needed anymore and dared to remove it during
refactoring.. This change reintroduces the check and adds it also to
other places that try to instanciate or resize a matrix.

When allocation of a matrix for an external range reference fails it is
checked whether the request is for such nasty entire rows or entire
columns ranges and if so tries to shrink the range to the actual data
area used, which at least in this case helps and works.

Additionally it fixes a long standing bug in the cached area to array
mapping which caused the cache to not be hit but instead the matrix was
aquired over and over again for the same range, causing a serious
performance bottle neck specifically for VLOOKUP and related functions
where the same external range is used in a lot of cells.

(cherry picked from commit 9e60bbdb3aa5f80ca80c9c4fdf7accd12c4a5d1c)

Change in convertToTokenArray() backported to older code flow.

disable means Enable(false), not true.. tdf#98389 related

EnableAdjustHeight(true) lead to the side effect that when clearing the
range of a pivot table during import, formulas in the same rows were
recalculated with then empty data. Through xSheetOp->clearContents() in
PivotTable::finalizeImport() of
sc/source/filter/oox/pivottablebuffer.cxx

In the case of the bugdoc attached to tdf#98389 that lead to all
VLOOKUP() calls resulting in #N/A errors because an empty string was
looked up.

(cherry picked from commit 088ba465d6bb0931c7034d564a3077e21078cb13)

e629ce871d255cc871671165c4da974f21041cec

Backported to ScMatrix implementation of 5-0.

setCellRangeData: check matrix size before operating, tdf#98389 follow-up

Now that we can have the one element error matrix we can't blindly call
ExecuteOperation() on it that would access elements out of bounds.

(cherry picked from commit 9cae3a88587deaf22ee83b2e36487013a1220a3b)

Backported to older code flow.

b6e5525f685c9324235f128a943dd716ea7a3fca

Change-Id: I35f67b48e665bf1cc8d672325c32b889f2e05c57
Reviewed-on: https://gerrit.libreoffice.org/23003
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Caolán McNamara <caolanm@redhat.com>
Tested-by: Caolán McNamara <caolanm@redhat.com>

1 files changed, 35 insertions, 8 deletions

diff --git a/sc/source/core/tool/scmatrix.cxx b/sc/source/core/tool/scmatrix.cxx
index c5c42f338ba5..328a4ba80166 100644
--- a/sc/source/core/tool/scmatrix.cxx
+++ b/sc/source/core/tool/scmatrix.cxx
@@ -2157,25 +2157,52 @@ void ScMatrix::DecRef() const
         delete this;
 }
 
-ScMatrix::ScMatrix( SCSIZE nC, SCSIZE nR) :
-    pImpl(new ScMatrixImpl(nC, nR)), nRefCnt(0)
+bool ScMatrix::IsSizeAllocatable( SCSIZE nC, SCSIZE nR )
 {
     SAL_WARN_IF( !nC, "sc", "ScMatrix with 0 columns!");
     SAL_WARN_IF( !nR, "sc", "ScMatrix with 0 rows!");
+    // 0-size matrix is valid, it could be resized later.
+    if ((nC && !nR) || (!nC && nR))
+    {
+        SAL_WARN( "sc", "ScMatrix one-dimensional zero: " << nC << " columns * " << nR << " rows");
+        return false;
+    }
+    if (nC && nR && (nC > (ScMatrix::GetElementsMax() / nR)))
+    {
+        SAL_WARN( "sc", "ScMatrix overflow: " << nC << " columns * " << nR << " rows");
+        return false;
+    }
+    return true;
+}
+
+ScMatrix::ScMatrix( SCSIZE nC, SCSIZE nR) :
+    pImpl(nullptr), nRefCnt(0)
+{
+    if (ScMatrix::IsSizeAllocatable( nC, nR))
+        pImpl = new ScMatrixImpl( nC, nR);
+    else
+        // Invalid matrix size, allocate 1x1 matrix with error value.
+        pImpl = new ScMatrixImpl( 1,1, CreateDoubleError( errStackOverflow));
 }
 
 ScMatrix::ScMatrix(SCSIZE nC, SCSIZE nR, double fInitVal) :
-    pImpl(new ScMatrixImpl(nC, nR, fInitVal)), nRefCnt(0)
+    pImpl(nullptr), nRefCnt(0)
 {
-    SAL_WARN_IF( !nC, "sc", "ScMatrix with 0 columns!");
-    SAL_WARN_IF( !nR, "sc", "ScMatrix with 0 rows!");
+    if (ScMatrix::IsSizeAllocatable( nC, nR))
+        pImpl = new ScMatrixImpl( nC, nR, fInitVal);
+    else
+        // Invalid matrix size, allocate 1x1 matrix with error value.
+        pImpl = new ScMatrixImpl( 1,1, CreateDoubleError( errStackOverflow));
 }
 
 ScMatrix::ScMatrix( size_t nC, size_t nR, const std::vector<double>& rInitVals ) :
-    pImpl(new ScMatrixImpl(nC, nR, rInitVals)), nRefCnt(0)
+    pImpl(nullptr), nRefCnt(0)
 {
-    SAL_WARN_IF( !nC, "sc", "ScMatrix with 0 columns!");
-    SAL_WARN_IF( !nR, "sc", "ScMatrix with 0 rows!");
+    if (ScMatrix::IsSizeAllocatable( nC, nR))
+        pImpl = new ScMatrixImpl( nC, nR, rInitVals);
+    else
+        // Invalid matrix size, allocate 1x1 matrix with error value.
+        pImpl = new ScMatrixImpl( 1,1, CreateDoubleError( errStackOverflow));
 }
 
 ScMatrix::~ScMatrix()