Skip to content
GitLab
Closed
Issue created by Administrator@rootContributor

DynamoRIO Incorrectly switch to thumb mode when encountering the MRRC instruction on armv7l

Created by: zenhumany

Describe the bug when i run the drrun, it crash like the following:

./drrun -debug  /usr/bin/sftp
<Starting application /usr/bin/sftp (10975)>
<Not tested @dynamorio/dynamorio/core/arch/emit_utils_shared.c:5574>
<Initial options = -no_dynamic_options -code_api -stack_size 56K -signal_stack_size 32K -max_elide_jmp 0 -max_elide_call 0 -early_inject -emulate_brk -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct >
<Paste into GDB to debug DynamoRIO clients:
set confirm off
add-symbol-file 'DynamoRIO-ARM-Linux-EABIHF-8.0.18929/lib32/debug/libdynamorio.so' 0x54b685a0
>
<(1+x) Handling our fault in a TRY at 0x54d31024>
<Invalid opcode encountered>
<Application /usr/bin/sftp (10975).  Application exception at PC 0x76f62f52.  
Signal 4 delivered to application as default action.
Callstack:
0x76f62f52   </usr/lib/libcrypto.so.1.0.0+0x119f52>
0xf8af04b0  
>
<Stopping application /usr/bin/sftp (10975)>
### Illegal instruction (core dumped)

The following is the cpu information :

Architecture: armv7l Byte Order: Little Endian CPU(s): 2 On-line CPU(s) list: 0,1 Thread(s) per core: 1 Core(s) per socket: 2 Socket(s): 1 Model name: ARMv7 Processor rev 10 (v7l) CPU max MHz: 996.0000 CPU min MHz: 396.0000

Some analysis

1. Switch to thumb mode incorrectly

The following is the output of DR with "loglevel 3" , in 53560 line ,DR switch to ARM mode, but at 53589 line , the DR use thumb decode the opcode.I think this is the first incorrectly place.

The IDA disassembly :

.text:0011993C                               ; void __fastcall __noreturn sub_11993C(int val)
.text:0011993C                               sub_11993C                              ; DATA XREF: OPENSSL_cpuid_setup+F0↑o
.text:0011993C                                                                       ; .text:off_3F9A8↑o
.text:0011993C 00 10 A0 E1                                   MOV             R1, R0  ; val
.text:00119940 08 00 9F E5                                   LDR             R0, =(unk_16B3A8 - 0x119950)
.text:00119944 10 40 2D E9                                   STMFD           SP!, {R4,LR}
.text:00119948 00 00 8F E0                                   ADD             R0, PC, R0 ; unk_16B3A8 ; env
.text:0011994C 24 97 FC EB                                   BL              siglongjmp
.text:0011994C                               ; End of function sub_11993C

The DR output :

53434 recreate_app : looking for 0x6b033198 in frag @ 0x6b033198 (tag 0x76f05a08) 
53435 recreate_app -- found valid state pc 0x76f05a08
53436         translation 0x76f05a08 is post-walk 0x00000000 so not fixing xsp
53437 recreate_app -- found ok pc 0x76f05a08
53438         restoring stolen register to 0x76f8a000
53439 recreate_app_state -- translation is:
53440 priv_mcontext_t @0x6aff49ac
......
53473         eflags = 0x600b0010
53474         pc     = 0x76f05a08
53475         translate_sigcontext: just set frame's eip to 0x76f05a08
53476 Got signal at pc 0x6b033198 in this fragment:
53477 Fragment 1412, tag 0x76f05a08, flags 0x1000030, shared, size 36:
53478 
53479   -------- prefix entry: --------
53480   0x6b033194  e59a0000   ldr    (%r10)[4byte] -> %r0
53481   -------- normal entry: --------
53482   0x6b033198  ec510f1e   mrrc   $0x0f $0x0e %c14 -> %r0 %r1
53483   0x6b03319c  e58a2008   str    %r2 -> +0x08(%r10)[4byte]
53484   0x6b0331a0  e1a0200e   mov    %lr -> %r2
53485   0x6b0331a4  eaffffff   b      $0x6b0331a8 <exit stub 0>
53486   -------- exit stub 0: -------- <target: 0x6af95180> type: ret
53487   0x6b0331a8  e58a1004   str    %r1 -> +0x04(%r10)[4byte]
53488   0x6b0331ac  e3041e10   movw   $0x00004e10 -> %r1
53489   0x6b0331b0  e3461b02   movt   $0x6b02 -> %r1[2byte]
53490   0x6b0331b4  e59af03c   ldr    +0x3c(%r10)[4byte] -> %pc
53491 
53492 Going to receive signal now
53493 get_sigstack_frame_ptr: using frame's xsp 0x7ec48a00
53494         placing frame at 0x7ec48700
53495 execute_handler_from_cache for signal 4
53496         xsp is 0x7ec48700
53497 copy_frame_to_stack: rt=0, src=0x6aff4c90, sp=0x7ec48700
53498 fixup_siginfo: updating si_addr from 0x6b033198 to 0x76f05a08
53499         converted sig=4 rt frame to non-rt frame
53500         copied frame from 0x6aff4c90 to 0x7ec48700
53501 blocked signals are now:
53502         1 = blocked
......
53559         64 = blocked
### 53560 Switching to ARM mode @0x76f0593c
53561         saved xax 0x00000004
53562         set next_tag to 0x76f0593c, resuming in fcache_return
53563 transfer_from_sig_handler_to_fcache_return
53564 sigcontext @0x6aff4d24:
53565         r0  =0x54de219c
53566         r1  =0x00000000
53567         r2  =0x76f574ac
53568         r3  =0x00000008
53569         r4  =0x76f57534
53570         r5  =0x76f573a8
53571         r6  =0x76f575b4
53572         r7  =0x7ec48b18
53573         r8  =0x00000001
53574         r9  =0x7ec48c0c
53575         r10 =0x6afc5000
53576         r11 =0x00000000
53577         r12 =0x76f57410
53578         sp  =0x7ec48700
53579         r14 =0x76c32800
53580         pc  =0x6af94ee0
53581         cpsr=0x600b0030
53582         set next_tag to handler 0x76f0593c, xsp to 0x7ec48700
53583         master_signal_handler 4 returning now to 0x6af94ee0
53584 
53585 Exit from asynch event
53586 
53587 d_r_dispatch: target = 0x76f0593c
53588 
### 53589 interp: start_pc = 0x76f0593c
### 53590   0x76f0593c  1000       asrs   %r0 $0x00000000 -> %r0
### 53591   0x76f0593e  e1a0       b      $0x76f05c82
53592 end_pc = 0x76f05940
53593 
53594 exit_branch_type=0x0 target=0x76f05c82 l->flags=0x1801
53595 Exit cti 0x6b0331c2 is targeting 0x6b0331c8 + 0x0 => 0x6b0331c8
53596 Fragment 1413, tag 0x76f0593c, flags 0x1400030, shared, size 32:
53597 
53598 Entry into F1413(0x76f0593c).0x6b0331c0 (T32)(shared)
53599 
53600 Exit from F1413(0x76f0593c).0x6b0331c2 (T32)(shared)
53601  (target 0x76f05c82 not in cache)
53602 
53603 d_r_dispatch: target = 0x76f05c82
53604 
53605 interp: start_pc = 0x76f05c82
53606   0x76f05c82  e12f       b      $0x76f05ee4
53607 end_pc = 0x76f05c84
53608 
53609 exit_branch_type=0x0 target=0x76f05ee4 l->flags=0x1801
53610 Exit cti 0x6b0331e4 is targeting 0x6b0331e8 + 0x0 => 0x6b0331e8
53611 Fragment 1414, tag 0x76f05c82, flags 0x1400030, shared, size 28:
53612 
53613 Entry into F1414(0x76f05c82).0x6b0331e4 (T32)(shared)
53614 
53615 Exit from F1414(0x76f05c82).0x6b0331e4 (T32)(shared)
53616  (target 0x76f05ee4 not in cache)
53617 
53618 d_r_dispatch: target = 0x76f05ee4
53619 
53620 interp: start_pc = 0x76f05ee4
53621   0x76f05ee4  0102       lsls   %r0 $0x00000004 -> %r2
53622   0x76f05ee6  b3a0       cbz    $0x76f05f52 %r0
53623 end_pc = 0x76f05ee8
53624 
53625 exit_branch_type=0x0 bb->exit_target=0x76f05ee8
53626 convert_to_near_rel: cbz/cbnz opcode
53627 exit_branch_type=0x11 target=0x76f05f52 l->flags=0x11
53628 exit_branch_type=0x0 target=0x76f05ee8 l->flags=0x1001
53629 Exit cti 0x6b033206 is targeting 0x6b033210 + 0x0 => 0x6b033210
53630 Exit cti 0x6b03320c is targeting 0x6b033224 + 0x0 => 0x6b033224
53631 Fragment 1415, tag 0x76f05ee4, flags 0x9400030, shared, size 56:
53632 
53633 Entry into F1415(0x76f05ee4).0x6b033204 (T32)(shared)
53634 
53635 Exit from F1415(0x76f05ee4).0x6b033206 (T32)(shared)
53636  (target 0x76f05f52 not in cache)
53637 
53638 d_r_dispatch: target = 0x76f05f52
53639 
53640 interp: start_pc = 0x76f05f52
53641 SYSLOG_WARNING: Invalid opcode encountered
53642 Invalid Thumb opcode @0x76f05f52: 0xebfc8002
53643 decode: invalid instr at 0x76f05f52
53644 Invalid Thumb opcode @0x76f05f52: 0xebfc8002
53645 decode: invalid instr at 0x76f05f52
53646   0x76f05f52  ebfc 8002  <INVALID>
53647 interp: invalid instr at 0x76f05f52
53648 record_pending_signal(4 at pc 0x76f05f52): signal is currently blocked
53649         action is not SIG_IGN
53650 blocked fatal signal 4 cannot be delayed: terminating
53651 execute_default_action for signal 4
53652 SYSLOG_WARNING: Application /usr/bin/sftp (19343).  Application exception at PC 0x76f05f52.
53653 Signal 4 delivered to application as default action.
53654 Callstack:
53657         0x76f05f52   </usr/lib/libcrypto.so.1.0.0+0x119f52>
53658         0xf8af04b0
53659 
53660 Terminating via kill
53661 SYSLOG_INFORMATION: Stopping application /usr/bin/sftp (19343)
53662 synch with all threads my id = 19343 Giving 4 permission and seeking 3 state
53663 add_process_lock: 0 lock 0x54eb2a7c: name=all_threads_synch_lock(mutex)@/home/runner/work/dynamorio/dynamorio/core/synch      .c:94

2. MRRC instruction lead to receare_app

the above switch to thumb mode incorrectly may be lead to by the mrrc instruction. MRRC instruction raised the SIGILL natively(not under DR). the following output is i run sftp natively under gdb.

./gdb-build/bin/gdb sftp
GNU gdb (GDB) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "arm-cortexa9_neon-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from sftp...
(No debugging symbols found in sftp)
(gdb) r
Starting program: /usr/bin/sftp 
### Program received signal SIGILL, Illegal instruction.
### 0x76f7ca08 in _armv7_tick () from /lib/libcrypto.so.1.0.0
(gdb) bt
#0  0x76f7ca08 in _armv7_tick () from /lib/libcrypto.so.1.0.0
#1  0x76ea2934 in OPENSSL_cpuid_setup () from /lib/libcrypto.so.1.0.0
#2  0x76fdefa8 in call_init (l=<optimized out>, argc=argc@entry=1, 
    argv=argv@entry=0x7efffd34, env=env@entry=0x7efffd3c) at dl-init.c:72
#3  0x76fdf104 in call_init (env=<optimized out>, argv=<optimized out>, 
    argc=<optimized out>, l=<optimized out>) at dl-init.c:30
#4  _dl_init (main_map=0x76fff958, argc=1, argv=0x7efffd34, env=0x7efffd3c) at dl-init.c:120
#5  0x76fcfb04 in _dl_start_user () from /lib/ld-linux-armhf.so.3
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
### (gdb) disassemble 0x76f7ca08
### Dump of assembler code for function _armv7_tick:
### => 0x76f7ca08 <+0>: mrrc 15, 1, r0, r1, cr14
###    0x76f7ca0c <+4>: bx lr
End of assembler dump.
(gdb) c
Continuing.
usage: sftp [-1246aCfpqrv] [-B buffer_size] [-b batchfile] [-c cipher]
          [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-l limit]
          [-o ssh_option] [-P port] [-R num_requests] [-S program]
          [-s subsystem | sftp_server] host
       sftp [user@]host[:file ...]
       sftp [user@]host[:dir[/]]
       sftp -b batchfile [user@]host
[Inferior 1 (process 11536) exited with code 01]
(gdb)

The following may useful information to diag this issue which is output by DR with --loglevel 3.

53317 interp: start_pc = 0x76e2b930
53318   0x76e2b930  eb036834   bl     $0x76f05a08 -> %lr
53319         NOT following direct call from 0x76e2b930 to 0x76f05a08
53320 end_pc = 0x76e2b934
53321 
53322 exit_branch_type=0x9 bb->exit_target=0x76f05a08
53323 exit_branch_type=0x9 target=0x76f05a08 l->flags=0x1809
53324 Exit cti 0x6b033178 is targeting 0x6b03317c + 0x0 => 0x6b03317c
53325 Fragment 1411, tag 0x76e2b930, flags 0x1000030, shared, size 36:
53326 
53327 Entry into F1411(0x76e2b930).0x6b033170 (A32)(shared)
53328 
53329 Exit from F1411(0x76e2b930).0x6b033178 (A32)(shared)
53330  (target 0x76f05a08 not in cache)
53331 
53332 d_r_dispatch: target = 0x76f05a08
53333 
### 53334 interp: start_pc = 0x76f05a08
### 53335   0x76f05a08  ec510f1e   mrrc   $0x0f $0x0e %c14 -> %r0 %r1
### 53336   0x76f05a0c  e12fff1e   bx     %lr
53337 end_pc = 0x76f05a10
53338 
53339 exit_branch_type=0x6 bb->exit_target=0x6af95180
53340 emit_fragment: bb use ibl <0x6af95180>
53341 exit_branch_type=0x6 target=0x6af95180 l->flags=0x1006
53342 Exit cti 0x6b0331a4 is targeting 0x6b0331a8 + 0x0 => 0x6b0331a8
53343 Fragment 1412, tag 0x76f05a08, flags 0x1000030, shared, size 36:
53344 
53345 Entry into F1412(0x76f05a08).0x6b033198 (A32)(shared)
53346 
53347 
### 53348 master_signal_handler: thread=19343, sig=4, xsp=0x6aff4c90, retaddr=0x00000004
### 53349 siginfo: sig = 4, pid = 1795371416, status = -1414812757, errno = 0, si_code = 1
53350         r0  =0x00000000
53351         r1  =0x00000000
53352         r2  =0x76f574ac
53353         r3  =0x00000008
53354         r4  =0x76f57534
53355         r5  =0x76f573a8
53356         r6  =0x76f575b4
53357         r7  =0x7ec48b18
53358         r8  =0x00000001
53359         r9  =0x7ec48c0c
53360         r10 =0x6afc5000
53361         r11 =0x00000000
53362         r12 =0x76f57410
53363         sp  =0x7ec48a00
53364         r14 =0x76e2b934
53365         pc  =0x6b033198
53366         cpsr=0x600b0010
53367 handle_nudge_signal: sig=4 code=1 errno=0
53368 record_pending_signal(4) from cache pc 0x6b033198
53369         not certain can delay so handling now
53370         action is not SIG_IGN
53371 add_process_lock: 0 lock 0x6af66dc0: name=synch_lock(mutex)@/home/runner/work/dynamorio/dynamorio/core/synch.c:144
53372 rank=10 owner=19343 owning_dc=0x6afaf040 contended_event=0xffffffff prev=0x00000000
53373 lock count_times_acquired=       1                              0                               0                                    0                               0+2 synch_lock(mutex)@/home/runner/work/dynamorio/dynamorio/core/synch.c:144
### 53374 translate context, thread 19343 at pc_recreatable spot translating
### 53375 recreate_app_state -- translating from:
53376 priv_mcontext_t @0x6aff49ac
53377         r0  = 0x00000000
53378         r1  = 0x00000000
53379         r2  = 0x76f574ac
53380         r3  = 0x00000008
53381         r4  = 0x76f57534
53382         r5  = 0x76f573a8
53383         r6  = 0x76f575b4
53384         r7  = 0x7ec48b18
53385         r8  = 0x00000001
53386         r9  = 0x7ec48c0c
53387         r10 = 0x6afc5000
53388         r11 = 0x00000000
53389         r12 = 0x76f57410
53390         r13 = 0x7ec48a00
53391         r14 = 0x76e2b934
53392         r15 = 0x6b033198
53393         q0  = 0x76f575b4 7ec48b18 00000001 76f70254
53394         q1  = 0x00000000 00000000 00000000 00000000
53395         q2  = 0x00000000 00000000 00000000 00000000
53396         q3  = 0x00000000 00000000 00000000 00000000
53397         q4  = 0x00000000 00000000 00000000 00000000
53398         q5  = 0x00000000 00000000 00000000 00000000
53399         q6  = 0x00000000 00000000 00000000 00000000
53400         q7  = 0x00000000 00000000 00000000 00000000
53401         q8  = 0x00001000 00000000 0000000c 00000000
53402         q9  = 0x0000000d 00000000 00000000 00000000
53403         q10 = 0x00000000 00000000 00000000 00000000
53404         q11 = 0x00000000 00000000 00000000 00000000
53405         q12 = 0x00000000 00000000 00000000 00000000
53406         q13 = 0x00000000 00000000 00000000 00000000
53407         q14 = 0x00000000 00000000 00000000 00000000
53408         q15 = 0x00000000 00000000 00000000 00000000
53409         eflags = 0x600b0010
53410         pc     = 0x6b033198
53411 
53412 building bb instrlist now *********************
53413 
53414 interp: start_pc = 0x76f05a08
53415   0x76f05a08  ec510f1e   mrrc   $0x0f $0x0e %c14 -> %r0 %r1
53416   0x76f05a0c  e12fff1e   bx     %lr
53417 end_pc = 0x76f05a10
53418 
53419 setting cur_pc (for fall-through) to 0x76f05a10
53420 exit_branch_type=0x6 bb->exit_target=0x6af95180
53421 
53422 done building bb instrlist *********************
53423 
53424 recreate_app : pc is in F1412(0x76f05a08)
53425 ilist for recreation:
53426 TAG  0x76f05a08
53427  +0    L3 @0x6afdffe8  ec510f1e   mrrc   $0x0f $0x0e %c14 -> %r0 %r1
53428  +4    m4 @0x6afdfea8  e58a2008   str    %r2 -> +0x08(%r10)[4byte]
53429  +8    m4 @0x6afdebe0  e58a2008   <label>
53430  +8    m4 @0x6afdf708  e1a0200e   mov    %lr -> %r2
53431  +12   L4 @0x6afdfe08  eafe82fc   b      $0x6af95180 <shared_bb_ibl_ret>
53432 END 0x76f05a08
53433 
### 53434 recreate_app : looking for 0x6b033198 in frag @ 0x6b033198 (tag 0x76f05a08)
### 53435 recreate_app -- found valid state pc 0x76f05a0853436         translation 0x76f05a08 is post-walk 0x00000000 so not fixing xsp
53437 recreate_app -- found ok pc 0x76f05a08
53438         restoring stolen register to 0x76f8a000
53439 recreate_app_state -- translation is:
53440 priv_mcontext_t @0x6aff49ac
53441         r0  = 0x00000000
53442         r1  = 0x00000000
53443         r2  = 0x76f574ac
53444         r3  = 0x00000008
53445         r4  = 0x76f57534
53446         r5  = 0x76f573a8
53447         r6  = 0x76f575b4
53448         r7  = 0x7ec48b18
53449         r8  = 0x00000001
53450         r9  = 0x7ec48c0c
53451         r10 = 0x76f8a000
53452         r11 = 0x00000000
53453         r12 = 0x76f57410
53454         r13 = 0x7ec48a00
53455         r14 = 0x76e2b934
53456         r15 = 0x76f05a08
53457         q0  = 0x76f575b4 7ec48b18 00000001 76f70254
53458         q1  = 0x00000000 00000000 00000000 00000000
53459         q2  = 0x00000000 00000000 00000000 00000000
53460         q3  = 0x00000000 00000000 00000000 00000000
53461         q4  = 0x00000000 00000000 00000000 00000000
53462         q5  = 0x00000000 00000000 00000000 00000000
53463         q6  = 0x00000000 00000000 00000000 00000000
53464         q7  = 0x00000000 00000000 00000000 00000000
53465         q8  = 0x00001000 00000000 0000000c 00000000
53466         q9  = 0x0000000d 00000000 00000000 00000000
53467         q10 = 0x00000000 00000000 00000000 00000000
53468         q11 = 0x00000000 00000000 00000000 00000000
53469         q12 = 0x00000000 00000000 00000000 00000000
53470         q13 = 0x00000000 00000000 00000000 00000000
53471         q14 = 0x00000000 00000000 00000000 00000000
53472         q15 = 0x00000000 00000000 00000000 00000000
53473         eflags = 0x600b0010
53474         pc     = 0x76f05a08
53475         translate_sigcontext: just set frame's eip to 0x76f05a08
53476 Got signal at pc 0x6b033198 in this fragment:
53477 Fragment 1412, tag 0x76f05a08, flags 0x1000030, shared, size 36:
53478 
53479   -------- prefix entry: --------
53480   0x6b033194  e59a0000   ldr    (%r10)[4byte] -> %r0
53481   -------- normal entry: --------
53482   0x6b033198  ec510f1e   mrrc   $0x0f $0x0e %c14 -> %r0 %r1
53483   0x6b03319c  e58a2008   str    %r2 -> +0x08(%r10)[4byte]
53484   0x6b0331a0  e1a0200e   mov    %lr -> %r2
53485   0x6b0331a4  eaffffff   b      $0x6b0331a8 <exit stub 0>
53486   -------- exit stub 0: -------- <target: 0x6af95180> type: ret
53487   0x6b0331a8  e58a1004   str    %r1 -> +0x04(%r10)[4byte]
53488   0x6b0331ac  e3041e10   movw   $0x00004e10 -> %r1
53489   0x6b0331b0  e3461b02   movt   $0x6b02 -> %r1[2byte]
53490   0x6b0331b4  e59af03c   ldr    +0x3c(%r10)[4byte] -> %pc
53491 
53492 Going to receive signal now
53493 get_sigstack_frame_ptr: using frame's xsp 0x7ec48a00
53494         placing frame at 0x7ec48700
53495 execute_handler_from_cache for signal 4
53496         xsp is 0x7ec48700
53497 copy_frame_to_stack: rt=0, src=0x6aff4c90, sp=0x7ec48700
53498 fixup_siginfo: updating si_addr from 0x6b033198 to 0x76f05a08
53499         converted sig=4 rt frame to non-rt frame
53500         copied frame from 0x6aff4c90 to 0x7ec48700
53501 blocked signals are now:
53502         1 = blocked
53503         2 = blocked
53504         3 = blocked
53505         4 = blocked
53506         6 = blocked
53507         9 = blocked
53508         10 = blocked
53509         12 = blocked

some other information

i have open a session in the dynamorio users forum in this link [](https://groups.google.com/g/dynamorio-users/c/E_4R3GbwhiA) ,they suggested that I file an issue here.