1<!DOCTYPE html> 2<html lang="en"> 3<head> 4<meta charset="UTF-8"> 5<meta name="viewport" content="width=device-width, initial-scale=1.0"> 6<link rel="icon" type="image/svg+xml" href="/favicon.svg"> 7<link href="/main.css?v=20260718-logo-spacing" rel="stylesheet" /> 8<title>Hacking Python</title> 9<!-- Highlight.js loads before the article to avoid a flash of raw code. --> 10<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/highlightjs/[email protected]/build/styles/default.min.css">
11<script src="https://cdn.jsdelivr.net/gh/highlightjs/[email protected]/build/highlight.min.js"></script>
vendor: 1 bytes, line 11
11
12<script src="https://cdn.jsdelivr.net/gh/highlightjs/[email protected]/build/languages/shell.min.js"></script>
vendor: 1 bytes, line 12
12
13<script src="https://cdn.jsdelivr.net/gh/highlightjs/[email protected]/build/languages/python.min.js"></script>
vendor: 1 bytes, line 13
13
14<script src="https://cdn.jsdelivr.net/gh/highlightjs/[email protected]/build/languages/c.min.js"></script>
vendor: 1 bytes, line 14
14
15<script defer src="/_vercel/insights/script.js"></script>
vendor: 1 bytes, line 15
15
16<script defer src="/_vercel/speed-insights/script.js"></script>
16 17</head> 18<body style="padding: 0px 5px; margin-top: 0;"> 19<div 20 style=" 21 display: flex; 22 flex-direction: column; 23 align-items: start; 24 justify-content: center; 25 gap: 10px; 26 width: 100%; 27 " 28> 29<nav class="nav-brand" aria-label="Primary navigation" style="margin-top: 10px;"><a class="nav-logo" href="/" aria-label="matmul"><img src="/images/logo.png" alt="Armillary sphere illustration by Albert Racinet (1888)"></a><span class="nav-links"><a href="/about.html">About</a><a href="https://www.youtube.com/@matmuls">YouTube</a><a href="https://x.com/@matmul">@matmul</a></span></nav> 30<div id="container"> 31 32<strong class="article-heading article-title">Hacking Python</strong> 33<i class="article-published">Published on 23.06.2025</i> 34<pre style=" 35 white-space: pre-wrap; 36 line-height: 1.5; 37 font-family: inherit; 38 font-size: inherit; 39"><span class="initial">I</span> came across this <a href="https://github.com/python/cpython/blob/f4911258a80409cb641f13578137475204ab43b5/Doc/howto/remote_debugging.rst#L39">document</a> on github which talks about the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">_PyRuntime</span> structure in cpython and how it can be used to profile or debug python code. I never knew about this structure before, so I thought it would be fun to mess around with it. 40 41<i>Note:</i> This uses a CPython <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">3.12.3</span> debug build on Linux. These internal structures can change across CPython versions and builds. 42 43<strong class="article-heading">Locating the Runtime State</strong> 44<code class="language-c">typedef struct pyruntimestate _PyRuntimeState;</code> 45So i compiled cpython <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">3.12.3</span> with debug flags and hooked it up with GDB. 46 47<code class="language-c">(gdb) ptype _PyRuntime 48type = struct pyruntimestate { 49 int _initialized; 50 int preinitializing; 51 int preinitialized; 52 int core_initialized; 53 int initialized; 54 _Py_atomic_address _finalizing; 55 struct pyinterpreterstate interpreters; 56 ... 57 PyInterpreterState _main_interpreter; 58} 59</code> 60In this CPython build, <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">_PyRuntime</span> contains the runtime state. This structure is placed in a dedicated <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">.PyRuntime</span> ELF section rather than the standard <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">.data</span> or <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">.bss</span> sections. This makes it easier to locate the runtime state directly through the section's virtual memory address (VMA). 61 62<code class="language-c">$ objdump -h /usr/bin/python3d 63 64/usr/bin/python3d: file format elf64-littleaarch64 65 66Sections: 67Idx Name Size VMA LMA File off Algn 68 0 .interp 0000001b 0000000000400270 0000000000400270 00000270 2**0 69 CONTENTS, ALLOC, LOAD, READONLY, DATA 70 ... 71 26 .PyRuntime 00074348 0000000000bf9150 0000000000bf9150 007e9150 2**3 72 CONTENTS, ALLOC, LOAD, DATA 73 ... 74</code> 75To find the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">_PyRuntime</span> struct, one has to add the base address of the python binary to the offset of the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">.PyRuntime</span> section but GDB makes it easy by providing the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">_PyRuntime</span> structure directly because it has the debug symbols loaded. 76 77A CPython process can have <a href="/$/outside-gil.html">one or more interpreters</a>, and each interpreter is represented by the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">PyInterpreterState</span> structure. The main interpreter is always present in the runtime state. Each interpreter has its own state, including the bytecode execution state. 78 79<code class="language-c">(gdb) ptype _PyRuntime.interpreters.main 80type = struct _is { 81 PyInterpreterState *next; 82 int64_t id; 83 ... 84 struct pythreads threads; 85 struct pyruntimestate *runtime; 86 ... 87 PyObject *sysdict; 88 PyObject *builtins; 89 struct _ceval_state ceval; 90 struct _import_state imports; 91 struct _gil_runtime_state _gil; 92 ... 93} * 94</code> 95The pattern is that there are a lot of structures that represent the state of the interpreter, threads, bytecode execution, etc. The <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">_PyRuntime</span> structure is the top-level structure that contains all these states. 96 97<strong class="article-heading">Walking the Frame Stack</strong> 98Now consider the following python code: 99 100<code class="language-python">import os 101import time 102import random 103 104def main(): 105 print(f"pid = {os.getpid()}") 106 107 salt = random.random() 108 rand = random.random() 109 110 secret = salt + rand 111 print(secret) 112 113 halt() 114 115def halt(): 116 while True: 117 pass 118 119if __name__ == '__main__': 120 main() 121</code> 122The goal is to read the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">
122secret</span> variable from the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">main()</span> function while the program is running. 123 124The hierarchy of structures for the current thread looks like this: 125 126<code class="language-c">_PyRuntime 127 ââ interpreters 128 ââ main 129 ââ threads 130 ââ head (current thread) 131 ââ cframe 132 ââ current_frame 133</code> 134In this program, the current frame is the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">halt()</span> function. We can use <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">previous</span> to go back to the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">main()</span> function. 135 136<code class="language-c">(gdb) p _PyRuntime.interpreters.main->threads.head->cframe.current_frame. 137f_builtins f_funcobj f_locals localsplus prev_instr return_offset 138f_code f_globals frame_obj owner previous stacktop 139</code> 140We can inspect a few things like the builtins, locals, globals, etc. 141 142<code class="language-c">(gdb) p _PyRuntime.interpreters.main->threads.head->cframe.current_frame.f_builtins 143$6 = {'__name__': 'builtins', '__doc__': "Built-in ... 144</code> 145<code class="language-c">(gdb) p _PyRuntime.interpreters.main->threads.head->cframe.current_frame.f_globals 146$7 = {'__name__': '__main__', ... , 'main': <function at remote 0xe29140a24f50>, 'halt': <function at remote 0xe2914095b410>} 147</code> 148<code class="language-c">(gdb) p _PyRuntime.interpreters.main->threads.head->cframe.current_frame.f_funcobj.ob_refcnt 149$11 = 2 150</code> 151The <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">f_code</span> field contains the metadata and the bytecode for the function. So in our case we can inspect a few things about the current function <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">halt()</span>. 152 153<code class="language-shell">(gdb) p _PyRuntime.interpreters.main->threads.head->cframe.current_frame.f_code. 154co_argcount co_localsplusnames 155co_code_adaptive co_name 156co_consts co_names 157co_exceptiontable co_ncellvars 158co_extra co_nfreevars 159co_filename co_nlocals 160co_firstlineno co_nlocalsplus 161co_flags co_posonlyargcount 162co_framesize co_qualname 163co_kwonlyargcount co_stacksize 164co_linetable co_version 165co_localspluskinds co_weakreflist 166(gdb) p _PyRuntime.interpreters.main->threads.head->cframe.current_frame.f_code.co_name 167$12 = 'halt' 168</code> 169<strong class="article-heading">Recovering the Secret</strong> 170So let's try to understand how the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">main()</span> is creating the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">secret</span> variable. 171 172<code class="language-python">salt = random.random() 173rand = random.random() 174secret = salt + rand 175</code> 176A simple salt added with a random number, if we inspect the bytecode using the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">dis</span> module, we can see that the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">secret</span> variable is created by adding the two random numbers. 177 178<code class="language-python">>>> import dis 179>>> dis.dis(main) 180 5 0 RESUME 0 181 182 ... 183 184 8 66 LOAD_GLOBAL 7 (NULL + random) 185 76 LOAD_ATTR 6 (random) 186 96 CALL 0 187 104 STORE_FAST 0 (salt) 188 189 9 106 LOAD_GLOBAL 7 (NULL + random) 190 116 LOAD_ATTR 6 (random) 191 136 CALL 0 192 144 STORE_FAST 1 (rand) 193 194 11 146 LOAD_FAST 0 (salt) 195 148 LOAD_FAST 1 (rand) 196 150 BINARY_OP 0 (+) 197 154 STORE_FAST 2 (secret) 198 199 ... 200 198 RETURN_CONST 0 (None) 201</code> 202Here the last <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">STORE_FAST</span> stores the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">
202secret</span> variable in the frame's <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">localsplus</span> array. The <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">f_localsplus</span> field in the frame structure contains the local variables and their values. 203 204You can also inspect <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">co_code_adaptive</span>, the bytecode stream CPython adapts at runtime. 205 206<code class="language-c">x/200xb _PyRuntime.interpreters.main->threads.head->cframe.current_frame->previous->f_code.co_code_adaptive 2070xe29140b836a0: 0x97 0x00 0x74 ... 208</code> 209<code class="language-python">>>> import dis 210>>> 211>>> addresses = [ 212... 0x97, 0x00, ... , 0x79, 0x00 213... ] 214>>> 215>>> for i, addr in enumerate(addresses): 216... base = 0xe29140b836a0 + i 217... print(f"0x{base:x}: 0x{addr:02x} -> {dis.opname[addr]}") 218... 219 220# 0xe29140b836a0: 0x97 -> RESUME 221# 0xe29140b836a1: 0x00 -> CACHE 222# 0xe29140b836a2: 0x74 -> LOAD_GLOBAL 223# ... 224# 0xe29140b83731: 0x01 -> POP_TOP 225# 0xe29140b83732: 0x58 -> <88> 226# 0xe29140b83733: 0x00 -> CACHE 227# 0xe29140b83734: 0x7c -> LOAD_FAST 228# 0xe29140b83735: 0x01 -> POP_TOP 229# 0xe29140b83736: 0x7a -> BINARY_OP 230# 0xe29140b83737: 0x00 -> CACHE 231# 0xe29140b83738: 0x01 -> POP_TOP 232# 0xe29140b83739: 0x00 -> CACHE 233# 0xe29140b8373a: 0x7d -> STORE_FAST 234... 235# 0xe29140b83766: 0x79 -> RETURN_CONST 236# 0xe29140b83767: 0x00 -> CACHE 237</code> 238Now we can read the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">secret</span> variable from the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">main()</span> function using the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">localsplus</span> field in the frame structure. 239 240<code class="language-c">(gdb) p *((PyFloatObject*)_PyRuntime.interpreters.main->threads.head->cframe.current_frame->previous->localsplus[0]) 241$25 = { ... , ob_fval = 0.34013542952956677} 242 243(gdb) p *((PyFloatObject*)_PyRuntime.interpreters.main->threads.head->cframe.current_frame->previous->localsplus[1]) 244$26 = { ... , ob_fval = 0.099622000445300007} 245 246(gdb) p *((PyFloatObject*)_PyRuntime.interpreters.main->threads.head->cframe.current_frame->previous->localsplus[2]) 247$27 = { ... , ob_fval = 0.43975742997486678} 248</code> 249You can verify that the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">secret</span> variable is the sum of the <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">salt</span> and <span style="font-family:'JetBrains Mono',monospace;font-size:15px;background:#F3F3F3;color:#D02F3A;padding:1px 3px">rand</span> variables. 250 251<code class="language-python">>>> 0.34013542952956677 + 0.099622000445300007 2520.4397574299748668 253</code> 254I didn't try to inject bytecode into interpreter frame directly, if you did let me know how it went (or) if you wanna discuss more on CPython internals. Feel free to reach out on <a href="https://x.com/matmul">twitter</a>.</pre> 255 256</div> 257</div> 258
259<script> 260 document.querySelectorAll('code[class^="language"]').forEach((block) => { 261 hljs.highlightElement(block); 262 }); 263</script>
263 264</body> 265</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.