Repository navigation
Conversation
A store hit now bails to the slow path instead of calling the GC barrier, mirroring into a metatable's MtCache, or growing properties. With no calls left the handler needs no stack frame: setf 6.8 -> 3.9 cyc, 58 -> 32 insts.
A small-integer key inside the array part is now handled without calls, so the handlers need no stack frame; any other key tail-calls the previous body. geti 4.3 -> 3.3 cyc (47 -> 26 insts), seti 8.0 -> 2.9 cyc (80 -> 29 insts); a string key pays the extra hop: hget 6.8 -> 8.0 cyc.
Decimal integers in string.format, tostring, concat and %q go through
format_into (int_format_into) into a stack buffer instead of a format!
String each; %o/%x use a local digit loop. format("%d,%d") 322 -> 166 cyc.
%a/%e/%f/%g now come from a port of lj_strfmt_num.c: an exact decimal bignum written straight into the output, with a rescale fast path for %e/%g below 14 digits. Ties are decided from the double itself, since the limbs past the rounding digit can be truncated (upstream rounds %.0e of 2.5e10 up), and %f of numbers below 2^-500 is zero outright (upstream wraps its limb ring there). %F is rejected, as in Lua, and an unknown conversion's error quotes the whole spec. tostring and concat now use %.14g like LuaJIT instead of 5.5's %.15g/%.17g round-trip. Float tostring 1500 -> 289 cyc.
A bare conversion skips parse_spec, literal runs are copied in one go,
precision zeros are a count instead of a staging buffer, the unpadded
case appends directly, and the output starts at 64 bytes.
format("%d,%d") goes from 166 to 142 cycles/op (LuaJIT -joff: 83).
The buffer lives in a userdata upvalue of format, as io keeps its state,
rather than in the VM. It is taken for the call, so a format re-entered
from __tostring gets a fresh one, and buffers past 64 KiB are freed.
format("%d,%d") goes from 142 to 119 cycles/op, %g and %.3f lose ~16.
A spec is spanned and checked as Lua's getformat/checkformat do, but
packed straight into an SFormat: the conversion's type, subtype and
accepted flags come from one match, and the modifiers are validated only
when there are any. Integers, strings, chars and pointers are written by
ports of lj_strfmt_putfxint/strfmt_putfstrlen/lj_strfmt_putfchar, which
replace FmtSpec and the old padding helpers, and plain %d stays a direct
NumBuffer write.
Matching Lua more closely along the way: a missing argument is reported
before the spec, spec errors come at the conversion's own point (after
the argument for integers and %e/%f/%g, after __tostring for %s), the
span runs to the conversion so '%5-d' and '%5%' are quoted whole, a
spec over 21 characters is 'invalid format (too long)', and %#o
prints one leading zero ('%#.3o' of 8 is 010, '%#.0o' of 0 is 0).
format("%d,%d") goes from 119 to 104 cycles/op (LuaJIT -joff: 83);
particles_bench 2400 frames is 2.14 Gcyc against 2.39 before this
branch and LuaJIT's 1.99.
Gc::new_with_trailing hands the trailing bytes to a callback instead of copying a slice, so a value can be followed by something other than bytes, and trailing_ptr / from_trailing_ptr convert between a Gc and the start of its trailing bytes. new_with_bytes is now a wrapper.
The array part moves from a Vec to a slots cell: a TrailingBytes cell of values that is never traced on its own. The table's trace marks it and traces its values, so the table's barrier covers stores into it, as with JSC's butterflies, and a replaced cell is left for the sweep instead of freed by the table's drop. First step towards tables without drop glue. Constructor arrays are a second GC object until they move into the table's own cell: ctor_arr +15%, ctor_arr10 +20%; geti/ipairs/len/particles within noise.
properties moves from a Vec to a slots cell holding the shape's slots, grown by doubling (from 4) on a property add that finds it full; the IC transition hit stores through push_property. Like the array part, the table traces it and its barrier covers stores. Creating objects costs a second GC object until the slots move into the table's own cell: ctor +18%, vnew +14%, setfield_new +14%; field loads and stores within noise.
GcAlloc hands out GC cells nothing traces and makes deallocation a no-op: the holder marks the cell (RawTable::allocation) when it is traced, and the sweep frees it after. The vendored RawTable loses its Drop impl, so with the array and named parts already in slots cells, TableState has no drop glue left, which is now asserted. dead_keys_are_not_retained counted only GC bytes, which now include the cleared table's own buckets; its baseline is a table as big whose keys own nothing. Benchmarks within noise (pack -3%).
A table cell now carries trailing values: its shape's inline slots, then
the array part a constructor sized, so a constructor's table is one
allocation. Slots past the inline capacity go to a spill cell. The
capacity is part of the shape: one root per size in INLINE_CAPS (0, 2,
4, 8, 16, 32), a template rooted at the size its fields need, and {} at
0 for now.
IC entries hold a SlotLoc resolved at fill time, a byte offset from the
table's lock for an inline slot or into the spill cell, so a hit reads
neither the shape's capacity nor the spill pointer for inline slots.
TableState reaches its own trailing values through
RefLock::trailing_ptr_of, which uses the provenance new_with_trailing
exposes.
Against the branch base: ctor -20%, vnew -17%, ctor_mt -17%, ctor_arr
-26%, ctor_arr10 -15%, particles -8%; field loads and stores within
noise (+1 to +3 instructions per hit). setfield_new (fields added to
{}) is +14% until {} gets a size hint.
A local initialized from a constructor, or from setmetatable(constructor,
mt), gets inline room for the distinct constant names that the following
statements of its block store into it (name.k = v, function name.k,
function name:k), up to a redeclaration of the name, as JSC's
StaticPropertyAnalyzer counts put_by_id into a new object. A rebound
setmetatable only costs the hint.
setfield_new -41%, class_new (setmetatable({}, C) then self.k = v) -35%,
module_tbl -19%; vnew moved by code placement only (equal with
-align-all-functions=6).
The integer, misc and dict-string parts move into one Owned cell, made by the first key that needs a hash part and traced through the table like a slots cell; dict mode is now the shape's is_dict. A table without hash keys pays one word for them instead of three RawTables, and TableState is 64 bytes (from 184). ctor -14%, vnew -17%, pack -10%, ctor_arr -9%, class_new -9%, particles -8%; hash lookups within noise.
rehash_ints counted each array key into its bin through memory, a store-to-load chain per key that took ~60% of a 1M-element append loop. Count non-nil values per power-of-two range instead, as LuaJIT's countarray does, and copy the old part with memcpy past 32 values. Appending 10M values: -31% cycles; 100-value tables: -19%; table.insert: -8%.
…48 bytes A table's metatable is now its shape's MtCache's back-pointer, which the cache leaves untraced: each table marks its metatable in its own trace, so a shape an inline cache holds still doesn't keep a metatable alive. A metatable's own MtCache moves into the aux cell (the hash parts' cell, made on adoption). Store ICs no longer check for a metatable on a hit; instead a site never caches a key MtCache::mirror watches. Header 64 -> 48 bytes, one malloc size class less for every table. ctor -5%, ctor_mt -3%, upd -5%; setmeta (a fresh metatable per object) +13%, as adopting a metatable now makes its aux cell.
nseq becomes an array part in the table's own cell, as a constructor's items do, capped at the largest array part a rehash makes (MAX_ASIZE): our nil is not zero bytes, so every slot is written. nrec picks the root shape whose inline capacity fits it. table.create(100) then 100 stores: -51% cycles; table.create(10): -70%; table.create(0, 4) then 4 fields: -41%.
A generic for gets a fourth hidden slot after the closing value, where TFORCALL keeps its position in a table it walks itself, as LuaJIT keeps its index in the control slot; the loop variables move up by one. The slot isn't counted toward the 200-local limit, so the limit matches lua. TFORCALL recognises the next that pairs returns and ipairs's iterator (kept in State) and steps without a call, taking TFORLOOP's jump too. Steps through the array part and string keys stay in the handler without a stack frame; tfor_next takes integer and other hash keys, tfor_ipairs the integer hash and __index. TableState::next_at walks in next's order by position: the part in bits 29-30, the index below, so positions stay small integers. Per element vs luajit -joff: pairs 73 -> 10.6 cycles (LJ 2.9), ipairs 36 -> 4.6 (LJ 9.9). particles 2400: 1.786 -> 1.647 Gcyc (LJ 2.01).
close_upvalues ran Vec::retain over every open upvalue of the thread whenever any was open, so each CLOSE, including the one at the end of every generic for, paid for all captured locals of every frame. Keep the list sorted by stack slot as Lua keeps openupval: a capture inserts in place, searching from the top, and a CLOSE checks the last entry and pops while it is at or above the level. Empty generic-for loop (next, e): 46.8 -> 36.6 cycles; pairs(e) 69.4 -> 55.9; ipairs(e) 49.0 -> 40.7.
TFORPREP checks the iterator and state, as LuaJIT's ISNEXT does, and marks the position slot: 0 to walk with next, -1 for ipairs, nil to call the iterator. TFORCALL dispatches on the mark alone instead of comparing the iterator each step, and next's first step no longer detours through tfor_next. The state is still checked to be a table, since debug.setlocal can change it; a changed iterator is not followed, as in LuaJIT. Testing the ipairs mark first keeps ipairs from paying a taken branch. vs the previous commit: empty next loop -11%, pairs(dict) -6%, pairs over a 10-field record -8%, ipairs per element -2%, an ipairs loop's setup +5% (the checks moved into TFORPREP).
pairs on a table without __pairs, and ipairs, now write their results straight into the call's slots from their own CALL entries, as math's builtins do, instead of going through the native-call machinery. Other call shapes (a TAILCALL, all results kept, another argument count) and a __pairs still go to the builtins. __pairs joins the metamethod list, so its presence is a bit in the metatable's MtCache, kept current like __tostring's; Symbols::pairs is now mm_pairs. Empty loop with pairs(e): 53.1 -> 36.4 cycles; ipairs(e): 45.7 -> 28.0; pairs over a 10-field record -5% per element.
next_at's positions now have five parts: array, integer keys, named slots, dict-mode string keys, other keys. Only one of the two string parts is ever non-empty (a shape-mode table's string hash part is empty, a dict-mode table's shape has no slots), so a step no longer loads the shape to ask which it is. Five parts take three bits: positions keep 2^28 per part. pairs over a 10-field record: -8% cycles; dict and ipairs unchanged; array part +2.6% (the part match orders its compares differently; testing the array part before the match instead moved the loss to records).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.