vcl: fix Win32 deadlocks from SolarMutexReleaser

To create and destroy thread-affine Win32 Windows and DCs, non-main
threads SendMessage() special messages like SAL_MSG_CREATEFRAME.

The main thread must handle these messages and return the result to
un-block the other thread.

This works fine as long as the main thread is in its message loop anyway
and blocked on GetMessage(); however if the main blocks trying to acquire
the SolarMutex that is held by the sending thread, deadlock results.

In order to work around this, there is some peculiar code in
ImplSalYieldMutexAcquireWithWait() to avoid blocking the main thread on
mpSalYieldMutex but instead block in GetMessage().

The crucial detail is that GetMessage() will immediately dispatch any
message sent via SendMessage(), which avoids the deadlock.

https://msdn.microsoft.com/en-us/library/windows/desktop/ms644936.aspx
https://msdn.microsoft.com/en-us/library/windows/desktop/ms644927.aspx

Most of the Win32 WndProc that acquire SolarMutex do so via
ImplSalYieldMutexAcquireWithWait(), but the main thread may also
temporarily drop SolarMutex and re-aquire it with the questionable
SolarMutexReleaser hack, which calls ImplSalAcquireYieldMutex()
instead, which blocks on mpSalYieldMutex.

Fix SolarMutexReleaser to call a new function
Application::ReAcquireSolarMutex() that does the right thing here:
acquire SolarMutex via ImplSalYieldMutexAcquireWithWait().

It turns out that this problem was already fixed before in commit
6a8fd4c76a969ac98d1aff91ff7442f43aee0006 but the problem was
insufficiently documented in the commit and it introduced the bug
that Application::Reschedule() was called without having the SolarMutex
locked, which caused timers to run without SolarMutex, so the commit
was reverted in 1ef1781390845d03b6e1518bbac81b818be62f3d.

Change-Id: I60aae555a9ee3c6390f584feddbc6b3cb7de0250

1 files changed, 7 insertions, 1 deletions

diff --git a/include/vcl/svapp.hxx b/include/vcl/svapp.hxx
index 78f758c6058f..7727b6e4c28a 100644
--- a/include/vcl/svapp.hxx
+++ b/include/vcl/svapp.hxx
@@ -504,6 +504,12 @@ public:
     */
     static void                 EndYield();
 
+    /** Acquire SolarMutex after it has been temporarily dropped completely.
+
+        This will Reschedule() on WNT and just acquire on other platforms.
+    */
+    static void                 ReAcquireSolarMutex(sal_uLong nReleased);
+
     /** @brief Get the Solar Mutex for this thread.
 
      Get the Solar Mutex that prevents other threads from accessing VCL
@@ -1687,7 +1693,7 @@ public:
 
     ~SolarMutexReleaser()
     {
-        Application::AcquireSolarMutex( mnReleased );
+        Application::ReAcquireSolarMutex(mnReleased);
     }
 };