Interpreter segfaults at random steps #49

AerScript Information

Aer Version (or commit ref): 2894c55dfa
Operating System: Linux
System Architecture (eg. arm, x86_64, ...): x86_64

Your problem description

AerScript interpreter segfaults at random steps during code execution (VM) when variables are declared. Problem is especially visible, when using loops. The more iterations, the more probability of segmentation fault. Whole 'typehinting' branch is affected.

Expected results

Execute script

Current results

Segmentation fault

class Program {
    void main() {
            int $c = 4;
            for(int $a = 0; $a < 10; $a++) {
                    for(int $b = 0; $b < 50; $b++) {
                            int $c = 1;
                            print("Hello world $c $b $a\n");
                    }
            }
            print("TEST: $c\n");
    }
}

Above code results in segmentation fault. However if I reduce the number of 2nd for loop iterations, script works as expected:

for(int $b = 0; $b < 5; $b++) {

The actual result:
Hello world 1 0 0
Hello world 1 1 0
Hello world 1 2 0
Hello world 1 3 0
Hello world 1 4 0
Hello world 1 5 0
Hello world 1 6 0
Hello world 1 7 0
Hello world 1 8 0
Hello world 1 9 0
Hello world 1 10 0
Hello world 1 11 0
Hello world 1 12 0
Hello world 1 13 0
Hello world 1 14 0
Hello world 1 15 0
Hello world 1 16 0
Hello world 1 17 0
Hello world 1 18 0
Hello world 1 19 0
Hello world 1 20 0
Hello world 1 21 0
Hello world 1 22 0
Hello world 1 23 0
Hello world 1 24 0
Hello world 1 25 0
Hello world 1 26 0
Hello world 1 27 0
Hello world 1 28 0
Hello world 1 29 0
Hello world 1 30 0
Hello world 1 31 0
Hello world 1 32 0
Hello world 1 33 0
Hello world 1 34 0
Hello world 1 35 0
Hello world 1 36 0
Hello world 1 37 0
Hello world 1 38 0
Hello world 1 39 0
Hello world 1 40 0
Hello world 1 41 0
Hello world 1 42 0
Hello world 1 43 0
Hello world 1 44 0
Hello world 1 45 0
Hello world 1 46 0
Hello world 1 47 0
Hello world 1 48 0
Hello world 1 49 0
Hello world 1 0 1
Hello world 1 1 1
Hello world 1 2 1
Hello world 1 3 1
Hello world 1 4 1
Hello world 1 5 1
Hello world 1 6 1
Hello world 1 7 1
Hello world 1 8 1
Hello world 1 9 1
Hello world 1 10 1
Segmentation fault

Script also works as expected when I comment out line 6:
int $c = 1;

Issue is hard to investigate, because attached script fails on trying to reallocate memory while formatting string that is going to be printed on the screen:

 (gdb) where
#0  MemBackendAlloc (pBackend=0x5555555c1f04, nBytes=32) at engine/lib/memory.c:142
#1  0x00007ffff7f54063 in MemBackendRealloc (pBackend=0x5555555c1f04, pOld=0x0, nBytes=16) at engine/lib/memory.c:182
#2  0x00007ffff7f541df in SyMemBackendRealloc (pBackend=0x5555555c1f04, pOld=0x0, nBytes=16) at engine/lib/memory.c:238
#3  0x00007ffff7f54fd7 in BlobPrepareGrow (pBlob=0x5555555c1eac, pByte=0x7fffffffaefc) at engine/lib/memory.c:722
#4  0x00007ffff7f55042 in SyBlobAppend (pBlob=0x5555555c1eac, pData=0x7fffffffb44e, nSize=1) at engine/lib/memory.c:736
#5  0x00007ffff7f535e8 in FormatConsumer (pSrc=0x7fffffffb44e, nLen=1, pData=0x7fffffffb4c0) at engine/lib/libfmt.c:663
#6  0x00007ffff7f53472 in InternFormat (xConsumer=0x7ffff7f53578 <FormatConsumer>, pUserData=0x7fffffffb4c0, zFormat=0x7ffff7fb73f2 "d", ap=0x7fffffffb530) at engine/lib/libfmt.c:626
#7  0x00007ffff7f536b3 in FormatMount (nType=6, pConsumer=0x5555555c1eac, xUserCons=0x0, pUserData=0x0, pOutLen=0x7fffffffb52c, zFormat=0x7ffff7fb73f0 "%qd", ap=0x7fffffffb530) at engine/lib/libfmt.c:698
#8  0x00007ffff7f538b2 in SyBlobFormat (pBlob=0x5555555c1eac, zFormat=0x7ffff7fb73f0 "%qd") at engine/lib/libfmt.c:726
#9  0x00007ffff7f8b89c in MemObjStringValue (pOut=0x5555555c1eac, pObj=0x5555555c1e94, bStrictBool=1 '\001') at engine/memobj.c:232
#10 0x00007ffff7f8c506 in PH7_MemObjToString (pObj=0x5555555c1e94) at engine/memobj.c:547
#11 0x00007ffff7f93e41 in VmByteCodeExec (pVm=0x5555555612cc, aInstr=0x555555580614, pStack=0x5555555c0cd4, nTos=-1, pResult=0x5555555c0584, pLastRef=0x7fffffffbea8, is_callback=0) at engine/vm.c:3115
#12 0x00007ffff7f98c7a in VmByteCodeExec (pVm=0x5555555612cc, aInstr=0x7fffffffc660, pStack=0x5555555c0584, nTos=2, pResult=0x7fffffffc730, pLastRef=0x0, is_callback=1) at engine/vm.c:5040
#13 0x00007ffff7f9bff6 in PH7_VmCallClassMethod (pVm=0x5555555612cc, pThis=0x5555555bd89c, pMethod=0x55555557645c, pResult=0x7fffffffc730, nArg=1, apArg=0x7fffffffc6f0) at engine/vm.c:6802
#14 0x00007ffff7f99897 in PH7_VmByteCodeExec (pVm=0x5555555612cc) at engine/vm.c:5321
#15 0x00007ffff7f89730 in ph7_vm_exec (pVm=0x5555555612cc, pExitStatus=0x7fffffffc7d8) at engine/api.c:892
#16 0x0000555555555612 in main ()

On the other side, when I remove all occurrences of print function, problem occurs in different area:

 (gdb) where
#0  0x00007ffff7f5155d in HashGetEntry (pHash=0x5555555766a4, pKey=0x55555557ccb4, nKeyLen=1) at engine/lib/dataset.c:194
#1  0x00007ffff7f515ed in SyHashGet (pHash=0x5555555766a4, pKey=0x55555557ccb4, nKeyLen=1) at engine/lib/dataset.c:214
#2  0x00007ffff7f90204 in VmExtractMemObj (pVm=0x5555555612cc, pName=0x7fffffffbda0, bDup=0, bCreate=0) at engine/vm.c:1419
#3  0x00007ffff7f927a0 in VmByteCodeExec (pVm=0x5555555612cc, aInstr=0x555555580614, pStack=0x5555555c0b54, nTos=-1, pResult=0x5555555c0364, pLastRef=0x7fffffffbea8, is_callback=0) at engine/vm.c:2510
#4  0x00007ffff7f98c7a in VmByteCodeExec (pVm=0x5555555612cc, aInstr=0x7fffffffc660, pStack=0x5555555c0364, nTos=2, pResult=0x7fffffffc730, pLastRef=0x0, is_callback=1) at engine/vm.c:5040
#5  0x00007ffff7f9bff6 in PH7_VmCallClassMethod (pVm=0x5555555612cc, pThis=0x5555555bd50c, pMethod=0x55555557645c, pResult=0x7fffffffc730, nArg=1, apArg=0x7fffffffc6f0) at engine/vm.c:6802
#6  0x00007ffff7f99897 in PH7_VmByteCodeExec (pVm=0x5555555612cc) at engine/vm.c:5321
#7  0x00007ffff7f89730 in ph7_vm_exec (pVm=0x5555555612cc, pExitStatus=0x7fffffffc7d8) at engine/api.c:892
#8  0x0000555555555612 in main ()

This time, it is trying to extract variable from hash map. Let's investigate this scenario:

(gdb) frame 0
#0  0x00007ffff7f5155d in HashGetEntry (pHash=0x5555555766a4, pKey=0x55555557ccb4, nKeyLen=1) at engine/lib/dataset.c:194
194                     if(pEntry->nHash == nHash && pEntry->nKeyLen == nKeyLen &&
(gdb) list
189             pEntry = pHash->apBucket[nHash & (pHash->nBucketSize - 1)];
190             for(;;) {
191                     if(pEntry == 0) {
192                             break;
193                     }
194                     if(pEntry->nHash == nHash && pEntry->nKeyLen == nKeyLen &&
195                                     pHash->xCmp(pEntry->pKey, pKey, nKeyLen) == 0) {
196                             return pEntry;
197                     }
198                     pEntry = pEntry->pNextCollide;
(gdb) print pEntry
$1 = (SyHashEntry_Pr *) 0x5555ffffffff
(gdb) print pEntry->nHash
Cannot access memory at address 0x555600000017

Apparently it throws segmentation fault on engine/lib/dataset.c:194 when it is trying to access pEntry->nHash. pEntry is a valid pointer, while nHash element contains address to some random memory area. All variables are being inserted to hash map by HashInsert() function defined at engine/lib/dataset.c:345 that is called from SyHashInsert defined at engine/lib/dataset.c:361. Whole structure is zeroed there and number of elements in hash map is set then in this structure member. I don't find a reason why it could fail.

Issue is hard to investigate, because attached script fails on trying to reallocate memory while formatting string that is going to be printed on the screen: (gdb) where #0 MemBackendAlloc (pBackend=0x5555555c1f04, nBytes=32) at engine/lib/memory.c:142 #1 0x00007ffff7f54063 in MemBackendRealloc (pBackend=0x5555555c1f04, pOld=0x0, nBytes=16) at engine/lib/memory.c:182 #2 0x00007ffff7f541df in SyMemBackendRealloc (pBackend=0x5555555c1f04, pOld=0x0, nBytes=16) at engine/lib/memory.c:238 #3 0x00007ffff7f54fd7 in BlobPrepareGrow (pBlob=0x5555555c1eac, pByte=0x7fffffffaefc) at engine/lib/memory.c:722 #4 0x00007ffff7f55042 in SyBlobAppend (pBlob=0x5555555c1eac, pData=0x7fffffffb44e, nSize=1) at engine/lib/memory.c:736 #5 0x00007ffff7f535e8 in FormatConsumer (pSrc=0x7fffffffb44e, nLen=1, pData=0x7fffffffb4c0) at engine/lib/libfmt.c:663 #6 0x00007ffff7f53472 in InternFormat (xConsumer=0x7ffff7f53578 <FormatConsumer>, pUserData=0x7fffffffb4c0, zFormat=0x7ffff7fb73f2 "d", ap=0x7fffffffb530) at engine/lib/libfmt.c:626 #7 0x00007ffff7f536b3 in FormatMount (nType=6, pConsumer=0x5555555c1eac, xUserCons=0x0, pUserData=0x0, pOutLen=0x7fffffffb52c, zFormat=0x7ffff7fb73f0 "%qd", ap=0x7fffffffb530) at engine/lib/libfmt.c:698 #8 0x00007ffff7f538b2 in SyBlobFormat (pBlob=0x5555555c1eac, zFormat=0x7ffff7fb73f0 "%qd") at engine/lib/libfmt.c:726 #9 0x00007ffff7f8b89c in MemObjStringValue (pOut=0x5555555c1eac, pObj=0x5555555c1e94, bStrictBool=1 '\001') at engine/memobj.c:232 #10 0x00007ffff7f8c506 in PH7_MemObjToString (pObj=0x5555555c1e94) at engine/memobj.c:547 #11 0x00007ffff7f93e41 in VmByteCodeExec (pVm=0x5555555612cc, aInstr=0x555555580614, pStack=0x5555555c0cd4, nTos=-1, pResult=0x5555555c0584, pLastRef=0x7fffffffbea8, is_callback=0) at engine/vm.c:3115 #12 0x00007ffff7f98c7a in VmByteCodeExec (pVm=0x5555555612cc, aInstr=0x7fffffffc660, pStack=0x5555555c0584, nTos=2, pResult=0x7fffffffc730, pLastRef=0x0, is_callback=1) at engine/vm.c:5040 #13 0x00007ffff7f9bff6 in PH7_VmCallClassMethod (pVm=0x5555555612cc, pThis=0x5555555bd89c, pMethod=0x55555557645c, pResult=0x7fffffffc730, nArg=1, apArg=0x7fffffffc6f0) at engine/vm.c:6802 #14 0x00007ffff7f99897 in PH7_VmByteCodeExec (pVm=0x5555555612cc) at engine/vm.c:5321 #15 0x00007ffff7f89730 in ph7_vm_exec (pVm=0x5555555612cc, pExitStatus=0x7fffffffc7d8) at engine/api.c:892 #16 0x0000555555555612 in main () On the other side, when I remove all occurrences of print function, problem occurs in different area: (gdb) where #0 0x00007ffff7f5155d in HashGetEntry (pHash=0x5555555766a4, pKey=0x55555557ccb4, nKeyLen=1) at engine/lib/dataset.c:194 #1 0x00007ffff7f515ed in SyHashGet (pHash=0x5555555766a4, pKey=0x55555557ccb4, nKeyLen=1) at engine/lib/dataset.c:214 #2 0x00007ffff7f90204 in VmExtractMemObj (pVm=0x5555555612cc, pName=0x7fffffffbda0, bDup=0, bCreate=0) at engine/vm.c:1419 #3 0x00007ffff7f927a0 in VmByteCodeExec (pVm=0x5555555612cc, aInstr=0x555555580614, pStack=0x5555555c0b54, nTos=-1, pResult=0x5555555c0364, pLastRef=0x7fffffffbea8, is_callback=0) at engine/vm.c:2510 #4 0x00007ffff7f98c7a in VmByteCodeExec (pVm=0x5555555612cc, aInstr=0x7fffffffc660, pStack=0x5555555c0364, nTos=2, pResult=0x7fffffffc730, pLastRef=0x0, is_callback=1) at engine/vm.c:5040 #5 0x00007ffff7f9bff6 in PH7_VmCallClassMethod (pVm=0x5555555612cc, pThis=0x5555555bd50c, pMethod=0x55555557645c, pResult=0x7fffffffc730, nArg=1, apArg=0x7fffffffc6f0) at engine/vm.c:6802 #6 0x00007ffff7f99897 in PH7_VmByteCodeExec (pVm=0x5555555612cc) at engine/vm.c:5321 #7 0x00007ffff7f89730 in ph7_vm_exec (pVm=0x5555555612cc, pExitStatus=0x7fffffffc7d8) at engine/api.c:892 #8 0x0000555555555612 in main () This time, it is trying to extract variable from hash map. Let's investigate this scenario: (gdb) frame 0 #0 0x00007ffff7f5155d in HashGetEntry (pHash=0x5555555766a4, pKey=0x55555557ccb4, nKeyLen=1) at engine/lib/dataset.c:194 194 if(pEntry->nHash == nHash && pEntry->nKeyLen == nKeyLen && (gdb) list 189 pEntry = pHash->apBucket[nHash & (pHash->nBucketSize - 1)]; 190 for(;;) { 191 if(pEntry == 0) { 192 break; 193 } 194 if(pEntry->nHash == nHash && pEntry->nKeyLen == nKeyLen && 195 pHash->xCmp(pEntry->pKey, pKey, nKeyLen) == 0) { 196 return pEntry; 197 } 198 pEntry = pEntry->pNextCollide; (gdb) print pEntry $1 = (SyHashEntry_Pr *) 0x5555ffffffff (gdb) print pEntry->nHash Cannot access memory at address 0x555600000017 Apparently it throws segmentation fault on engine/lib/dataset.c:194 when it is trying to access pEntry->nHash. pEntry is a valid pointer, while nHash element contains address to some random memory area. All variables are being inserted to hash map by HashInsert() function defined at engine/lib/dataset.c:345 that is called from SyHashInsert defined at engine/lib/dataset.c:361. Whole structure is zeroed there and number of elements in hash map is set then in this structure member. I don't find a reason why it could fail.

Without doubt, the problem is connected with variables declaration. Because the interpreter throws segmentation fault at random conditions and debugger describes a problem under different circumstances, I suspect that variable might be unnecessarily popped out from the stack.

All types of loops are affected by this problem.

Fixed with 6b18e204cd.