# Deadlock with fftw, shared libs and Windows
**URL:** https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494
**Category:** Engineering
**Created:** [December 8, 2017, 8:35am UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494 "2017-12-08T08:35:22Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@simon.rit](https://discourse.itk.org/u/simon.rit)
#### Post date: [December 8, 2017, 8:35am UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/1 "2017-12-08T08:35:23Z")
Hi,
I have a deadlock at the end of my programs when I use FFTW, shared libs and MSVC. You can see it here:
[http://my.cdash.org/viewTest.php?onlyfailed&buildid=1335053](http://my.cdash.org/viewTest.php?onlyfailed&buildid=1335053).
Note that I don’t have this problem under linux:
[my.cdash.org/index.php?project=RTK&date=2017-12-08](http://my.cdash.org/index.php?project=RTK&date=2017-12-08)
After recompiling FFTW with cmake in debug mode, I nailed it down to a deadlock in the function [kill\_workforce](https://github.com/FFTW/fftw3/blob/62edb203fc09c8c8ac2c2d5ac3299ea8d4dc7838/threads/threads.c#L351). I guess the problem can also come from FFTW but I know they don’t want to answer Windows questions. Has anyone encountered such an issue and would you have any suggestion on how to debug this? My feeling is that it comes from a destruction order issue…
Thanks in advance,
Simon
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [December 8, 2017, 3:17pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/2 "2017-12-08T15:17:38Z")
This sounds pretty much like the thread pool destructor deadlock [bug](https://discourse.itk.org/t/threadpool-hanging-in-dlls/426) from last month.
---
### Author:  [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [December 8, 2017, 3:45pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/3 "2017-12-08T15:45:42Z")
@dzenanz As I recall that bug occurred after the program exited the main, and resource were trying to be freed properly.
@simon.rit Can you get the stack trace with the program is in deadlock?
---
### Author:  [@simon.rit](https://discourse.itk.org/u/simon.rit)
#### Post date: [December 8, 2017, 3:56pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/4 "2017-12-08T15:56:12Z")
I can. I don’t have it with me but it was after in main, in fftw, and in
[https://github.com/InsightSoftwareConsortium/ITK/blob/30206c0ec0cfecb6df9dec22e9a54e98463189db/Modules/Filtering/FFT/src/itkFFTWGlobalConfiguration.cxx#L552](https://github.com/InsightSoftwareConsortium/ITK/blob/30206c0ec0cfecb6df9dec22e9a54e98463189db/Modules/Filtering/FFT/src/itkFFTWGlobalConfiguration.cxx#L552)
on the ITK side. I’ll send the full stack on Monday if you need it.
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [December 8, 2017, 4:02pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/5 "2017-12-08T16:02:20Z")
Thread pool bug in short: the threads get killed by Windows before the destructor gets called, so destructor hangs waiting for non-existing threads to signal the semaphore.
The exact same thing seems to be going on here. As you are getting us the stack trace, you might as well confirm the above hypothesis. Pause the program, look in Threads (ctrl-alt-H on VisualStudio) and see if there is only one left.
---
### Author:  [@simon.rit](https://discourse.itk.org/u/simon.rit)
#### Post date: [December 11, 2017, 10:47am UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/6 "2017-12-11T10:47:00Z")
Here is the stack
[External Code]
> fftw3.dll!os\_sem\_down(void \* \* s) Line 206 C
> fftw3.dll!kill\_workforce() Line 373 C
> fftw3.dll!fftw\_threads\_cleanup() Line 458 C
> fftw3.dll!fftw\_cleanup\_threads() Line 65 C
> ITKFFT-4.12.dll!itk::FFTWGlobalConfiguration::~FFTWGlobalConfiguration() Line 557 C++
> [External Code]
> ITKCommon-4.12.dll!itk::LightObject::UnRegister() Line 165 C++
> ITKCommon-4.12.dll!itk::Object::UnRegister() Line 447 C++
> ITKFFT-4.12.dll!itk::SmartPointeritk::FFTWGlobalConfiguration::UnRegister() Line 170 C++
> ITKFFT-4.12.dll!itk::SmartPointeritk::FFTWGlobalConfiguration::~SmartPointeritk::FFTWGlobalConfiguration() Line 67 C++
> [External Code]
And there is only one thread indeed. It’s not obvious how to apply a similar strategy as what has been done in the ITK patch since they do not even record the thread address… Any suggestion?
---
### Author:  [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [December 11, 2017, 11:40am UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/7 "2017-12-11T11:40:52Z")
Does fftw have a method you can use to manually cleanup?
edit: correct auto-correct
---
### Author:  [@simon.rit](https://discourse.itk.org/u/simon.rit)
#### Post date: [December 11, 2017, 12:08pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/8 "2017-12-11T12:08:32Z")
In FFTW? I guess that’s the purpose of fftw\_cleanup\_threads().
---
### Author:  [@simon.rit](https://discourse.itk.org/u/simon.rit)
#### Post date: [December 11, 2017, 1:54pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/9 "2017-12-11T13:54:26Z")
With this additional line in threads/threads.c of FFTW3, it seems to work:
```auto
diff --git a/threads/threads.c b/threads/threads.c
index 9f40363..dca28ef 100644
--- a/threads/threads.c
+++ b/threads/threads.c
@@ -367,6 +367,7 @@ static void kill_workforce(void)
worker_queue = q->cdr;
q->w = &w;
os_sem_up(&q->ready);
+ if (WaitForSingleObject(q->ready, 0) == WAIT_TIMEOUT)
os_sem_down(&termination_semaphore);
unmake_worker(q);
}
```
I guess I will report it to FFTW to let them find a suitable implementation when this is confirmed unless someone has a better suggestion.
---
### Author:  [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [December 12, 2017, 9:23pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/10 "2017-12-12T21:23:56Z")
Have you tried adding a call to [fftw\_cleanup\_threads](http://www.fftw.org/fftw3_doc/Usage-of-Multi_002dthreaded-FFTW.html) to manually clean up the fftw threads before exiting of the main?
---
### Author:  [@simon.rit](https://discourse.itk.org/u/simon.rit)
#### Post date: [December 13, 2017, 1:39pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/11 "2017-12-13T13:39:30Z")
I just tried. Yes, it works: if I add fftw\_cleanup\_threads() and fftwf\_cleanup\_threads() before main ends, there is no deadlock. However, I’m not sure I fully understand your comment: it seems to me that this is not a solution if you don’t want to ask every ITK user who uses FFTW + shared libs + Windows to add these lines at the end of their main. Is there any other solution to let ITK do these cleanups automatically as was meant by the FFTWGlobalConfiguration destructor?
Or do you advise me to do a pull request with my patch on FFTW?
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [December 13, 2017, 1:55pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/12 "2017-12-13T13:55:25Z")
I think it is better to do a pull request to FFTW, as the problem is there. If we only fix it in ITK, the other users of FFTW might still suffer (or have to resort to static libraries).
---
### Author:  [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [December 13, 2017, 2:08pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/13 "2017-12-13T14:08:02Z")
My suggestion is a work around to enable your code to work with current ITK and current FFTW and not depend on a patch version of either. That can be a valuable solution depending on the requirements.
I agree that ideally, the user should not have to worry about this.
---
### Author:  [@simon.rit](https://discourse.itk.org/u/simon.rit)
#### Post date: [December 13, 2017, 3:08pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/14 "2017-12-13T15:08:39Z")
Thanks for the suggestion. What do you think of not calling fftw\_cleanup\_threads() in FFTWGlobalConfiguration in this configuration (I guess depending on ITKFFT\_EXPORT)? I can prepare an ITK patch if you’d agree.
I’ll work on a PR for FFTW meanwhile.
---
### Author:  [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [December 13, 2017, 3:13pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/15 "2017-12-13T15:13:00Z")
This thread deadlocking issue appears to be a new thing. Do we know what VS version or OS, or libraries are required to reproduced it? Or has is always been there?
---
### Author:  [@simon.rit](https://discourse.itk.org/u/simon.rit)
#### Post date: [December 13, 2017, 3:19pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/16 "2017-12-13T15:19:33Z")
I don’t know (I was not handling the Windows compilation of [RTK](http://www.openrtk.org) before but I guess FFTW was not activated in ITK). The bug occurs with Visual Studio 12 2013 Win64 on Windows 7 (see ctest [script file](http://my.cdash.org/viewNotes.php?buildid=1336013)). I could try to do other MSVC if this is really important.
---
### Author:  [@dzenanz](https://discourse.itk.org/u/dzenanz)
#### Post date: [December 13, 2017, 3:36pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/17 "2017-12-13T15:36:56Z")
ITK’s thread pool bug was occurring on VS2013 and VS2017. But judging by Windows official documentation, it should happen not matter which compiler you are using, much less the exact version of it.
---
### Author:  [@blowekamp](https://discourse.itk.org/u/blowekamp)
#### Post date: [December 13, 2017, 3:39pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/18 "2017-12-13T15:39:36Z")
Perhaps it has only been revealed (edit) recently because of the way ITK has changed to use shared libraries.
@simon.rit I don’t think there is much harm is remove the proper thread shutdown for FFT for the case of shared libraries (ITK\_BUILD\_SHARED\_LIBS), and MSVC.
---
### Author:  [@fbudin](https://discourse.itk.org/u/fbudin)
#### Post date: [December 22, 2017, 3:43pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/19 "2017-12-22T15:43:34Z")
@simon.rit’s patch has been integrated in ITK. When Updating FFTW in ITK, one will have to verify that this problem also has been solved in FFTW, so carry over this patch in the new FFTW.
---
### Author:  [@simon.rit](https://discourse.itk.org/u/simon.rit)
#### Post date: [December 22, 2017, 4:02pm UTC](https://discourse.itk.org/t/deadlock-with-fftw-shared-libs-and-windows/494/20 "2017-12-22T16:02:49Z")
If some people can explain the problem to the FFTW community, I encourage you to participate to the conversation on FFTW github: [https://github.com/FFTW/fftw3/pull/121](https://github.com/FFTW/fftw3/pull/121).