tdf#148669: do not try to transform tail to unit rectangle

... just to transform it back when it's used.

The problem was that the transform had zero vertical scale; thus
it couldn't be inverted. The unchanged matrix was used both in the
SdrCaptionPrimitive2D constructor, and in its create2DDecomposition,
thus scaling the X coordinate up twice. The result, which depends
on the original value of X, was huge for comments attached to cells
with large column numbers. Trying to convert such coordinate from
pixels to logic units exceeded 64-bit integer range.

Avoiding the transformations back and forth is possible, because
the transform is constant in the object. This also avoids unneeded
overhead.

There is an open question why there was a need in the calculations
for the XLSX file. The problem doesn't happen with ODS, so likely
there is a calculation of the draw page size in some circumstances
that may be avoided.

Change-Id: Iccbe77936ec1078daf6b05b0b2d44a6fa3536c4f
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/133217
Tested-by: Mike Kaganski <mike.kaganski@collabora.com>
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>

1 files changed, 2 insertions, 11 deletions

diff --git a/svx/source/sdr/primitive2d/sdrcaptionprimitive2d.cxx b/svx/source/sdr/primitive2d/sdrcaptionprimitive2d.cxx
index 507f9ae0f4e3..a371a75f6106 100644
--- a/svx/source/sdr/primitive2d/sdrcaptionprimitive2d.cxx
+++ b/svx/source/sdr/primitive2d/sdrcaptionprimitive2d.cxx
@@ -76,7 +76,7 @@ namespace drawinglayer::primitive2d
                     createHiddenGeometryPrimitives2D(
                         false,
                         basegfx::B2DPolyPolygon(getTail()),
-                        getTransform()));
+                        {}));
             }
             else
             {
@@ -89,11 +89,9 @@ namespace drawinglayer::primitive2d
                         getSdrLFSTAttribute().getLine(),
                         attribute::SdrLineStartEndAttribute()));
 
-                aTransformed = getTail();
-                aTransformed.transform(getTransform());
                 aRetval.push_back(
                     createPolygonLinePrimitive(
-                        aTransformed,
+                        getTail(),
                         getSdrLFSTAttribute().getLine(),
                         getSdrLFSTAttribute().getLineStartEnd()));
             }
@@ -132,13 +130,6 @@ namespace drawinglayer::primitive2d
             mfCornerRadiusX(fCornerRadiusX),
             mfCornerRadiusY(fCornerRadiusY)
         {
-            // transform maTail to unit polygon
-            if(getTail().count())
-            {
-                basegfx::B2DHomMatrix aInverse(getTransform());
-                aInverse.invert();
-                maTail.transform(aInverse);
-            }
         }
 
         bool SdrCaptionPrimitive2D::operator==(const BasePrimitive2D& rPrimitive) const