Regarding C++11 noexcept

1

It was an unpleasant surprise to me, to read here at the forum (December 2017) that noexcept might sometimes harm the performance. As noted by @matt.mccormick at warning: dynamic exception specifications are deprecated in C++11

Now I’m just posted this question to Microsoft’s Developer Community forum: C++11 noexcept implementation still a performance loss with VS2017? Hope that Visual C++ is doing better by now…

I do have some good news, already :slight_smile: I just profiled the following code on Visual Studio 2017 (15.9.5), Release:

std::vector<itk::NeighborhoodAllocator<int>> neighborhoodAllocators;

itk::NeighborhoodAllocator<int> neighborhoodAllocator;
neighborhoodAllocator.set_size(1);
neighborhoodAllocator[0] = 42;

neighborhoodAllocators.assign(7500000, neighborhoodAllocator);

neighborhoodAllocators.reserve(neighborhoodAllocators.capacity() + 1);

I observed that the last line of code, calling std::vector::reserve, runs more than four times faster now that itk::NeighborhoodAllocator has a ITK_NOEXCEPT move-constructor:

itk::NeighborhoodAllocator move-constructor without ITK_NOEXCEPT: ~0.6 second
itk::NeighborhoodAllocator move-constructor with ITK_NOEXCEPT: ~0.13 second :tada: :tada: :tada:

The ITK_NOEXCEPT itk::NeighborhoodAllocator move-constructor that I proposed was merged onto the ITK git master branch by @phcerdan, earlier this week:

So when a class has a manually implemented non-throwing move-constructor that is faster than its copy-constructor, the noexcept specifier is certainly beneficial. :slight_smile:

3 Likes
2

Check https://developercommunity.visualstudio.com/content/problem/425370/c11-noexcept-implementation-still-a-performance-lo.html for the replies by @Terry Mahaffey :slight_smile:

2 Likes
3

It appears to me that three categories of non-throwing functions could be distinguished:

  1. Those where adding a noexcept specifier does evidently improve the runtime performance, in a way that is observable, reproducible, and logically explainable.

  2. Those where adding noexcept appears neutral to the performance: in these cases, no significant effect has been observed.

  3. Those where adding noexcept actually harms the performance.

Would it be a good idea to use the C++11 noexcept specifier directly, instead of the ITK_NOEXCEPT macro, for the first category of functions?

I would like to propose to use noexcept, instead of ITK_NOEXCEPT, for those functions for which the performance gain is obvious. For example, the move-constructor and the move-assignment operator of itk::NeighborhoodAllocator.

In cases where the performance gain is not (yet) obvious, ITK_NOEXCEPT could still be used.

What do you think?

4

Thank you for your work on trying to figure this out.

How many compilelers have you tested on to make this conclusion?

Another case of note is where I added the old no throw was in the base objects and command classes in methods related to the destructor and managing reference between objects. As I recall if an exception occurs here it can lead to leaks or worse yet invalid referenced.

2 Likes
5

ITK_NOEXCEPT expands to noexcept already.
Check the generated file in /path/build/Modules/Core/Common/itk_compiler_detection.h

#  if defined(ITK_COMPILER_CXX_NOEXCEPT) && ITK_COMPILER_CXX_NOEXCEPT
#    define ITK_NOEXCEPT noexcept
#    define ITK_NOEXCEPT_EXPR(X) noexcept(X)
#  else
#    define ITK_NOEXCEPT
#    define ITK_NOEXCEPT_EXPR(X)
#  endif

I guess ITK 5.0 doesn’t have to worry about compilers that do not support noexcept, so we could use noexcept directly.

But there is no difference between them.

ITK_NOEXCEPT is not an optional qualifier that the user can switch off if you meant that.

2 Likes
6

@Niels_Dekker as you have noted and Terry Mahaffey notes in the reply, the behavior depends on the specific situation and the compiler. And, in many cases, the compiler can infer whether exception-related optimizations are possible or not.

@jhlegarreta and I are working towards a GitHub check that will show performance differences across the different platforms and compilers.

Once we have the check in place, we will be able to determine 1., 2., and 3. We have observed that behavior varies vastly across compliers sometimes. Once we have performance testing, we could add the specification for 1. remove it for ‘2.’ and ‘*3.’ If we find that Visual Studio just has issues in some cases where the other compilers see a performance increase, we could define ITK_NOEXCEPT to be empty with Visual Studio. Where it always demonstrates peformance improvements, we could use noexcept.

Until then, ITK_NOEXCEPT serves a useful, grep’able marker that is still functional.

1 Like
7

@matt.mccormick Thanks for your explanation! I’m very interested to hear about any preliminary result from the GitHub check you and @jhlegarreta are working towards.

So far I haven’t seen any example for which adding noexcept would actually harm the performance. Did you, already?

A complete example to check the performance of neighborhoodAllocators.reserve is at:

In this case, it appears that std::vector::reserve became more than 4x faster when a noexcept move-constructor was added to itk::NeighborhoodAllocator. But I only tried VS2017 (Version 15.9.5) Release x64. I’m very interested to hear if the performance gain from noexcept can be reproduced with other compilers.

1 Like
8

The performance gain of flagging noexcept a move constructor will be noticed by all the compilers, because it is at the std library level. Not related with any internals of compilers. The std algorithms choose the move (usually faster) only if it is noexcept or if the compiler can infer that is noexcept.

The harm at performance because flagging noexcept ANY function happens with the VS compiler, because its internal implementation as you were answered by the VS developer. This is a low-level detail. The performance increase at the algorithm level will be much greater than this implementation-detail drawback. But also help us to not go crazy flagging everything noexcept.

My 2 cents and take away for us would be: let’s put noexcept only where matters for the std algorithms to choose the fastest route. This is at move and swaps. And only if the compiler cannot infer it by default.

2 Likes
9

I have not seen a lot of evidence in any direction – putting my science hat on, I would definitely like to see more evidence :tophat::microbe::microscope: .

This is great! :heart:

After testing on GCC / Linux and Apple / Clang, it would be reasonable to mark noexcept.

10

Personally I find item 14 from Scott Meyers, Effective Modern C++ (2015) quite convincing: “Declare functions noexcept if they won’t emit exceptions”. If you don’t have access to the book, there’s a draft version of the item at https://aristeia.com/EC++11-14/noexcept%202014-03-31.pdf

But honestly, I just got myself some evidence that noexcept might indeed harm the performance, in some very specific cases, when using Visual C++. I could reproduce this issue with the latest version of VS2017 (version 15.9.5), Release configuration, but only for 32-bit (Win32 x86). I could not reproduce the issue for 64-bit.

For a large number of function calls (numberOfFuncCalls greater than 10,000,000), I compared

for (int i = 0; i < numberOfFuncCalls; ++i)
{
  NoexceptFunc(false);
}

for (int i = 0; i < numberOfFuncCalls; ++i)
{
  UnspecifiedFunc(false);
}

The functions NoexceptFunc(bool) and UnspecifiedFunc(bool) were equivalent, except that the first one had a noexcept specifier. To avoid inlining, I put both of them in a DLL, separate from the main application. NoexceptFunc(bool) was implemented as follows:

__declspec(dllexport) void NoexceptFunc(bool doThrowException) noexcept
{
  assert(!doThrowException);
  ThrowIf(doThrowException);
}

And ThrowIf(doThrowException) was as follows:

void ThrowIf(const bool doThrowException)
{
  if (doThrowException) throw std::exception{};
}

It appeared to me that the function call UnspecifiedFunc(false) was twice as fast as NoexceptFunc(false)!

If there’s any interest, I can post the entire example on my github.

1 Like
11

Please have a look at my noexcept benchmark: https://github.com/N-Dekker/noexcept_benchmark (under construction). I hope it can be of help to the ITK/GitHub check that you are working towards, @matt.mccormick and @jhlegarreta It currently has six test cases:

  1. inline function calls
  2. exported library function calls
  3. recursive function calls
  4. catching function calls
  5. template recursion
  6. std::vector<my_string> reserve

The benchmark runs on GitHub CI, being compiled by MSVC, Clang, and GCC. It produces output like this:

[Linux GCC][`throw` included][catching function calls (N = 12000)]
    noexcept       implicit    (durations in seconds)
    0.00004000  <  0.00008880
    0.00003880  <  0.00004660
    0.00003740  <  0.00004470
    0.00003740  <  0.00004480
    0.00003730  <  0.00004490
    0.00003730  <  0.00004490
    0.00003730  <  0.00004490
    0.00003730  <  0.00004460
    0.00003730  <  0.00004490
    0.00007500  >  0.00004620
    ----------     ----------
    0.00041510  <  0.00049530  (sum of durations)
    0.00003730  <  0.00004460  (shortest durations)
Ratio sum of durations noexcept/implicit: 0.83807624
Ratio sum of durations implicit/noexcept: 1.19320887
In this case, neither implicit nor noexcept specifications always appear faster.

Unfortunately, the output from various compilers and configurations is still a bit hard to find, somewhere within the CmdLine log of Azure, for example at https://dev.azure.com/dekkerware/dekkerware/_build/results?buildId=41 Please let me know if you have a suggestion how to make the output easier visible. My knowledge about yaml/yml is very limited!

12

@Niels_Dekker wow, awesome! :clap: :broccoli: :rocket:

We now have a team of four students from the University of North Carolina-Chapel Hill working on an ITK performance visualization application . We should expect that it will take three or four months before we will be able to use it with these benchmarks, but these benchmarks will be helpful by providing data-driven understanding of performance and allow us to make real improvements.

1 Like
13

In the meantime I did a major revision to the noexcept benchmark. Latest tag: https://github.com/N-Dekker/noexcept_benchmark/tree/v03

This is the revised set of test cases:

  1. inline function calls
  2. exported library function calls
  3. catching function calls
  4. inc ++ and dec --
  5. stack unwinding
  6. std::vector<my_string> reserve

The results can now be found a little bit easier, simply by clicking “Benchmark results…” at https://dev.azure.com/dekkerware/dekkerware/_build/results?buildId=50

It appears that with VS2017 64-bit, test case 4, 5, and 6 benefit significantly from noexcept.

With GCC, test case 2, 3, 5, and 6 appear to benefit from noexcept.

With Clang, only test case 6 appears to benefit from noexcept.

Please have a look at the benchmark results and tell me what you think!

3 Likes
14

@Niels_Dekker nice job !

The ITK performance benchmarking relies on JSON files produced by the benchmarking scripts. A sample of the generated JSON files is available in data.kitware.com.

Generalizing the module-level performance ecosystem to specific class-level or even more fine-grained, lower-level analysis may require further effort.

2 Likes
15

:+1:! Well done @Niels_Dekker.

The benchmarks show that we get different results depending on the compiler and usage.

It will be interesting to see results for ITK usage.

1 Like
16

Interesting, thanks, Jon! I think it would be possible for noexcept_benchmark to generate similar JSON files. Currently, the noexcept benchmark consists of 6 test cases. Each test case is run 10 times, for both noexcept and implicitly defined exception specifications. How many JSON files should then be generated?

Indeed, Matt! Unfortunately!

Regarding the test cases:

Test case 3 (catching function calls) checks something like this:

try { f(); } catch(const std::exception&) { std::cerr << "Oops!"; }

If f() is noexcept, the catch clause can be eliminated by the compiler, during optimization, which would improve the performance.

Test case 4 (inc ++ and dec --) checks something like this:

++v; f(); --v;

If f() is noexcept, the compiler may reorder the instructions to ++v; --v; f();, and then just remove the first two instructions!

Test case 5 (stack unwinding) should check the effect of stack unwinding no longer being required, when an exception is thrown within a function that is noexcept:

void f() noexcept
{
  MyClassType stackObject;
  g(); // if g() throws an exception, stackObject does not need to be destructed!
}

Please let me know if you have any more comment about the test cases!

1 Like
17

Sorry for not fully grocking the test result. Did ‘noexcept’ ever hurt performance?

18

Unfortunately, yes, @blowekamp. ‘noexcept’ did in some specific test cases hurt the performance. Especially for test case 1 (inline function calls) and 2 (exported library function calls), when doing 32-bit compilation (x86) of VS2017:

 [MSVC 32-bit][`throw` included][inline function calls (N = 567890000)]
 noexcept       implicit    (durations in seconds)
 0.22582380  >  0.00000040
 ...
 0.21689120  >  0.00000040
 ----------     ----------
 2.16602530  >  0.00001590  (sum of durations)
 0.20901280  >  0.00000030  (shortest durations)
 Ratio sum of durations noexcept/implicit: 136228.00628931
 Ratio sum of durations implicit/noexcept: 0.00000734
 In this case, implicit exception specifications always appear faster.

 [MSVC 32-bit][`throw` included][exported library function calls (N = 98765432)]
 noexcept       implicit    (durations in seconds)
 0.41370900  >  0.22174150
 ...
 0.42939760  >  0.20602690
 ----------     ----------
 4.43308230  >  2.27357290  (sum of durations)
 0.40841380  >  0.20602690  (shortest durations)
 Ratio sum of durations noexcept/implicit: 1.94983073
 Ratio sum of durations implicit/noexcept: 0.51286503
 In this case, implicit exception specifications always appear faster.

From https://dev.azure.com/dekkerware/dekkerware/_build/results?buildId=50 (Windows Job), Benchmark results 32-bit.

19

For the module case, each JSON file contains a list of values (Probes:Values), since the benchmark is run a number of times (Probes:Iterations) for a given module for the same configuration, the same as you did ! Also, statistics such as the mean time (Probes:Mean), the maximum (Probes:Maximum), and the minimum (Probes:Minimum) are computed and written to the file.

So all that information could be contained in a single benchmark JSON file.

HTH.

3 Likes
20

Last Monday I did a lightning talk of 5 minutes, “noexcept considered harmful???”, at C++ on Sea, in England, presenting preliminary results of my noexcept benchmark. And of course, I mentioned ITK :slight_smile: My PowerPoint file (five slides) is now at https://nd.home.xs4all.nl/cpp/talks/NielsDekker-CppOnSea-Feb-2019-noexcept.pptx

3 Likes