tdf#115483 svl xmloff sc sw: verify all ODF 1.2 protection-key hashes

ODF 1.2 has added some mandatory requirements for protection-key hashes
which did not exist in ODF 1.1.

This affects sections and indexes in ODT documents, as well as
spreadsheets and sheets in ODS documents.

1. Accept the following hashed passwords:
    * UTF16 LE/BE encoded StarOffice-SHA1, OOo legacy and allowed by ODF 1.1
    * UTF8 encoded proper SHA1, as required by ODF 1.2
    * UTF8 encoded SHA256, as required by ODF 1.2
        - specified either with the wrong URL used in the ODF 1.2 spec
          or the correct URL from the W3C spec, see OFFICE-3702
    * Excel+SHA1 double-hash, only in Calc, see OFFICE-2112

2. Round-trip any of the above as-is
    * for SHA256 only write the URL from the ODF 1.2 spec

3. Generate only UTF16 LE encoded SHA1 for now, so that older LO
   releases can still verify the password
    * some time in the future, switch to generating some valid ODF 1.2 hash

More changes are necessary in Calc, which can actually preserve
different hashes for the same passwords in its runtime data model,
whereas Writer just has a single buffer without even any metadata.

For the Calc unit tests we need one document per hash because the
protection-key attribute can be set on the entire spreadsheet, which
is an unique element in the document.

There are further uses of SvlPasswordHelper for change-tracking passwords,
but apparently those are stored in settings.xml, so ODF has no
requirements for them, so let's leave that as it is.

Reviewed-on: https://gerrit.libreoffice.org/49352
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Michael Stahl <mstahl@redhat.com>
(cherry picked from commit 398275ba9f4d65bebcc78864e70eee6212a84397)

Change-Id: Icb720b14ae9c0d9c04d2e082769ae2b74e3af8aa
Reviewed-on: https://gerrit.libreoffice.org/49390
Tested-by: Jenkins <ci@libreoffice.org>
Reviewed-by: Eike Rathke <erack@redhat.com>

1 files changed, 45 insertions, 7 deletions

diff --git a/svl/source/misc/PasswordHelper.cxx b/svl/source/misc/PasswordHelper.cxx
index 00558f6a1630..48aa165507ee 100644
--- a/svl/source/misc/PasswordHelper.cxx
+++ b/svl/source/misc/PasswordHelper.cxx
@@ -19,11 +19,32 @@
 
 
 #include <svl/PasswordHelper.hxx>
+#include <comphelper/hash.hxx>
 #include <rtl/digest.h>
 #include <memory>
 
 using namespace com::sun::star;
 
+void SvPasswordHelper::GetHashPasswordSHA256(uno::Sequence<sal_Int8>& rPassHash, OUString const& rPassword)
+{
+    OString const tmp(OUStringToOString(rPassword, RTL_TEXTENCODING_UTF8));
+    ::std::vector<unsigned char> const hash(::comphelper::Hash::calculateHash(
+        reinterpret_cast<unsigned char const*>(tmp.getStr()), tmp.getLength(),
+        ::comphelper::HashType::SHA256));
+    rPassHash.realloc(hash.size());
+    ::std::copy(hash.begin(), hash.end(), rPassHash.begin());
+}
+
+void SvPasswordHelper::GetHashPasswordSHA1UTF8(uno::Sequence<sal_Int8>& rPassHash, OUString const& rPassword)
+{
+    OString const tmp(OUStringToOString(rPassword, RTL_TEXTENCODING_UTF8));
+    ::std::vector<unsigned char> const hash(::comphelper::Hash::calculateHash(
+        reinterpret_cast<unsigned char const*>(tmp.getStr()), tmp.getLength(),
+        ::comphelper::HashType::SHA1));
+    rPassHash.realloc(hash.size());
+    ::std::copy(hash.begin(), hash.end(), rPassHash.begin());
+}
+
 void SvPasswordHelper::GetHashPassword(uno::Sequence<sal_Int8>& rPassHash, const sal_Char* pPass, sal_uInt32 nLen)
 {
     rPassHash.realloc(RTL_DIGEST_LENGTH_SHA1);
@@ -74,14 +95,31 @@ bool SvPasswordHelper::CompareHashPassword(const uno::Sequence<sal_Int8>& rOldPa
 {
     bool bResult = false;
 
-    uno::Sequence<sal_Int8> aNewPass(RTL_DIGEST_LENGTH_SHA1);
-    GetHashPasswordLittleEndian(aNewPass, sNewPass);
-    if (aNewPass == rOldPassHash)
-        bResult = true;
-    else
+    if (rOldPassHash.getLength() == RTL_DIGEST_LENGTH_SHA1)
+    {
+        uno::Sequence<sal_Int8> aNewPass(RTL_DIGEST_LENGTH_SHA1);
+        GetHashPasswordSHA1UTF8(aNewPass, sNewPass);
+        if (aNewPass == rOldPassHash)
+        {
+            bResult = true;
+        }
+        else
+        {
+            GetHashPasswordLittleEndian(aNewPass, sNewPass);
+            if (aNewPass == rOldPassHash)
+                bResult = true;
+            else
+            {
+                GetHashPasswordBigEndian(aNewPass, sNewPass);
+                bResult = (aNewPass == rOldPassHash);
+            }
+        }
+    }
+    else if (rOldPassHash.getLength() == 32)
     {
-        GetHashPasswordBigEndian(aNewPass, sNewPass);
-        bResult = (aNewPass == rOldPassHash);
+        uno::Sequence<sal_Int8> aNewPass;
+        GetHashPasswordSHA256(aNewPass, sNewPass);
+        bResult = aNewPass == rOldPassHash;
     }
 
     return bResult;