Replace the add+ldar with a ldr+dmb pair in HT lookup and results in -3% to -28% (reduction) in overhead (instrument vs. native) ratios of SPECInt 2006 and 2017. Although, found a regression in 657.xz_s model (+7%).
An add+ldar pair was used to prevent memory-access instructions from being reordered to ensure a hash mask is always loaded before a hash_table is loaded. The same ordering restriction is also imposed in the corresponding update_lookuptable_tls() routine.
With the ldr+dmb pair replacement, an add instruction is eliminated from the critical path and contributed majorly to overhead reductions.
In addition, a frequently taken branch of the inner-loop epilogue is converged into a more condensed (4 instr.) and smaller inner-loop, with the cost of adding 1 more sub instruction in both 2 exit paths.
Since exit paths are not as hot as HT lookup, the impact of adding a sub instruction in both is quite trivial (not seen).
In order to condense the inner-loop down to 4 instructions, a pre-index ldr is replaced with a post-index version in the prologue.
On the hit-exit path, register use is carefully swapped and results in a move instruction elimination. The number of instructions on the path remains unchanged since a sub instruction is added to its entry.
Tests: SPECInt 2006 and 2017 ran on ARM Juno r2 (w/ 8GB RAM, Debian)