32-bit native_exec_retakeover loses control

This is another regression in native_exec from the hybrid work.

After adding proper checks for losing control in the common.nativeexec* tests, we see that the common.nativeexec_retakeover test for 32-bit loses control and fails.

It seems to not recognize any transitions from the app to the applib as calls. It does go native, but doesn't mangle the retaddr. It seems to regain control on the ELF hooks for retakeover. At the end though it does not regain control.

32-bit default ops:

  0x08049698  c7 04 24 39 00 00 00 mov    $0x00000039 -> (%esp)[4byte]
  0x0804969f  e8 3c f7 ff ff       call   $0x08048de0 %esp -> %esp 0xfffffffc(%esp)[4byte]
<...>
  0x08048de0  ff 25 30 d0 04 08    jmp    0x0804d030[4byte]
<...>
Entry into F102(0xf77bb4c0).0x4da6dc68 (shared)
Exit from sourceless ibl: bb ret        
 (target 0xf778a16c not in cache)
fragment_add_ibl_target tag 0xf778a16c, branch 0, F0 
         Table ret_trace, table 0x4dab5240, mask 0x0000007f
         Table indcall_trace, table 0x4dab5680, mask 0x0000007f
         Table indjmp_trace, table 0x4dab5ac0, mask 0x0000007f

dispatch: target = 0xf778a16c
at native_exec target: checking TOS 0xfff88c60 => 0x080496a4 for retaddr
native_exec: decoding @0x0804969e looking for call
native_exec: decoding @0x0804969f looking for call
native_exec: found call @ pre-*TOS 0x0804969f
module libcommon.nativeexec.appdll.so is on native list, executing natively

32-bit -native_exec_retakeover:

Entry into F102(0xf77c94c0).0x46299c68 (shared)
Exit from sourceless ibl: bb ret        
 (target 0xf779816c not in cache)
fragment_add_ibl_target tag 0xf779816c, branch 0, F0 
         Table ret_trace, table 0x462f1240, mask 0x0000007f
         Table indcall_trace, table 0x462f1680, mask 0x0000007f
         Table indjmp_trace, table 0x462f1ac0, mask 0x0000007f

dispatch: target = 0xf779816c
module libcommon.nativeexec.appdll.so is on native list, executing natively

The call from app to applib goes through dl fixup and on 64-bit that ends in jmp*, but on 32-bit it ends in ret (uses ret as a jmp*).

There is added code in interp.c at_native_exec_gateway by the comment "Is this a return from a non-native module into a native module?" which looks for a ret (if native_exec_retakeover is on only) and sets is_call to false. I bet for the hybrid work that added this code, we only ran 64-bit, and this test couldn't tell if we lost control on 32-bit.

The fix is to move those checks down below the "guessing" code.