Files
carbon-lang/testing
Chandler Carruth 571f46adc5 Make file_test dramatically faster. (#3834)
Our testing had started to take more noticable time so I looked at where
the time went to see if it could be improved easily. Almost all of the
time was spent building regex matchers. That code path allocates a lot
of memory and does a lot of processing in order to make the regex fast
to run over the input. While that's not really a great tradeoff for how
we use these matchers, the bigger issue is that we don't need a regex is
the vast majority of cases.

This change inspects the pattern more deeply to avoid most of the work.

First, it looks for the
cases where the pattern doesn't require any adjustment at all and
directly builds a matcher from that if it can. This avoids an extra
string allocation entirely. This is probably only a minor improvement,
but it is also very easy.

The larger change is to process the string in two phases. First, we
expand the keywords and check for a regex region. If we find no regex
region, we can directly build a string equality matcher for the expanded
string. This still allocates an extra copy of the string but is
*dramatically* cheaper that building a regex.

Finally, if we *do* find a regex, we re-process the string to escape
everything and transform the regex sequence into its valid form before
building the matcher.

The result for me is an over 5x reduction in test time. Profiling
afterward shows a lot more opportunities for optimization here if things
get slow again. The actual toolchain code isn't really visible in the
profile yet.

Note, I was profiling the normal build, which has asserts and ASan and
such. I've not looked at the optimized build.
2024-03-29 21:07:26 +00:00
..