ast2nif

Search:
Group by:
Source   Edit  

AST to NIF bridge.

Types

DecodeContext = object
Source   Edit   
LoadFlag = enum
  LoadFullAst, AlwaysLoadInterface
Source   Edit   
ModuleSuffix = distinct string
Source   Edit   
ParsedSymName = object
  name*: string
  module*: string
  count*: int
Source   Edit   
PrecompiledModule = object
  topLevel*: PNode
  deps*: seq[ModuleSuffix]
  logOps*: seq[LogEntry]
  module*: PSym
  reexportedModules*: seq[(string, string)]
  genericOffers*: seq[tuple[generic, inst: PSym, concreteTypes: seq[PType],
                            genericParamsCount: int]] ## generic instances this module created; modulegraphs.nim rebuilds
                                                      ## `procInstCache` from them so a consumer reuses the instance instead of
                                                      ## re-instantiating it in its own (operator-blind) module scope.
  typeOffers*: seq[tuple[generic: PSym, inst: PType]] ## generic TYPE instances this module created; modulegraphs.nim rebuilds
                                                      ## `typeInstCache` from them so a consumer reuses the baked instance
                                                      ## (e.g. a `mixin`/`compiles()`-dependent array bound) instead of
                                                      ## re-instantiating it with a different bound in its own scope.
  includes*: seq[string]
Source   Edit  

Consts

BackendLocalMarker = "@bk"
Suffix marker for a PROCESS-LOCAL backend-minted entity (a closure :env type/obj/field/hidden-param minted while the VM compiles a routine body to run a macro). Such entities have no stable cross-process identity, so each module that references one emits its OWN module-local def named …<thisModuleSuffix>@bk and the loader homes it to the reading module with a backendItemId (disjoint from real ids). See transf.transformBody. Reserved module-suffix sentinel for module-less magic singleton types — the nil type is created via newSysType with the graph idgen, whose module can be -1 (e.g. during VM const-eval before a real module is current), so its uniqueId.module is unresolvable. Such a type has no fields and an identity that is fully captured by its kind, so we serialize it with this sentinel and reconstruct it on load (see createTypeStub) without ever touching a .nif file. A real moduleSuffix never starts with '@'. Source   Edit   
SysModuleSuffix = "@sys"
Source   Edit  

Procs

proc createDecodeContext(config: ConfigRef; cache: IdentCache): DecodeContext {.
    ...raises: [], tags: [], forbids: [].}
Supposed to be a global variable Source   Edit   
proc firstSon(n: Cursor): Cursor {.inline, ...raises: [], tags: [], forbids: [].}
Non-consuming peek at the first child of a TagLit. The inc is on a copy, so it never advances the caller's cursor. Source   Edit   
proc getMainModuleSuffix(c: DecodeContext): string {.inline, ...raises: [],
    tags: [], forbids: [].}
Source   Edit   
proc globalName(sym: PSym; config: ConfigRef): string {....raises: [],
    tags: [RootEffect], forbids: [].}
Source   Edit   
proc icNifTypeName(typ: PType; config: ConfigRef): string {....raises: [],
    tags: [RootEffect], forbids: [].}
The serialized NIF name of a type, recorded next to RTTI data definitions in the cnif artifact so a later run can re-demand the typeinfo when a reused TU still references it (the def-retention check). Backend-minted types have no NIF name. Source   Edit   
proc isStub(s: PSym): bool {.inline, ...raises: [], tags: [], forbids: [].}
Source   Edit   
proc isStub(t: PType): bool {.inline, ...raises: [], tags: [], forbids: [].}
Source   Edit   
proc loadedModuleTypes(c: var DecodeContext; module: FileIndex): seq[PType] {.
    ...raises: [KeyError, OSError, IOError],
    tags: [ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}
Stubs for every TYPE this module owns — but, unlike before, WITHOUT force- loading them. writeLoweredModule emits a real def into the .t.bif only for the ones already Complete (= the lower stage actually loaded, hence possibly MUTATED — lambda-lifting flips a proc type to ccClosure and grows env types with captured fields). Every untouched type stays Partial, so a reference to it serializes as a SymUse that a cg/emit consumer resolves from the .s.bif (loader fallback typeCursor/ensureSemBuf) — the lower stage leaves those defs unchanged, so re-emitting them into .t.bif was pure cost. Collect names first: createTypeStub may register modules / mutate c.types, which must not invalidate the index iterator. Source   Edit   
proc loadNifModule(c: var DecodeContext; f: FileIndex;
                   interf, interfHidden: var TStrTable;
                   flags: set[LoadFlag] = {}): PrecompiledModule {.
    ...raises: [OSError, IOError, KeyError, ValueError, Exception],
    tags: [RootEffect, ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}
Source   Edit   
proc loadNifModule(c: var DecodeContext; suffix: ModuleSuffix;
                   interf, interfHidden: var TStrTable;
                   flags: set[LoadFlag] = {}): PrecompiledModule {.
    ...raises: [OSError, IOError, KeyError, ValueError, Exception],
    tags: [ReadEnvEffect, ReadIOEffect, ReadDirEffect, RootEffect], forbids: [].}
Source   Edit   
proc loadSym(c: var DecodeContext; s: PSym) {.
    ...raises: [KeyError, OSError, IOError, Exception, ValueError],
    tags: [RootEffect, ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}
Source   Edit   
proc loadType(c: var DecodeContext; t: PType) {.
    ...raises: [KeyError, OSError, IOError, Exception],
    tags: [RootEffect, ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}
Source   Edit   
proc moduleSymbolStubs(c: var DecodeContext; module: FileIndex): seq[PSym] {.
    ...raises: [KeyError, OSError, IOError],
    tags: [ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}

Stubs for every non-type symbol serialized in module's NIF index. The per-module backend uses this to emit the routines a module OWNS: procs are serialized as (sd ...) symbol-defs and loaded lazily, never as nkProcDef statements in the top-level stmt list, so genTopLevelStmt alone never reaches them — without this, a routine called only from other modules would be emitted by nobody once the demanding module merely prototypes it.

Returns lazy stubs: the index table is moved out while iterating (loading a symbol can register new modules and invalidate the iterator), so the caller forces full load (.kind, .ast) and filters AFTER this returns, with the index back in place.

Source   Edit   
proc nextBackendSymItem(c: var DecodeContext; module: int32): int32 {.
    ...raises: [KeyError], tags: [], forbids: [].}
Allocate the next backend-minted SYM item for module from the SAME per-module counter the loader uses when it re-homes @bk syms loaded from the module's .t.bif (loadSymStub/extractLocalSymsFromTree). The lower stage serializes its lifted hooks/temps as @bk syms, and cg mints MORE backend syms (RTTI destroy wrappers, ...) into the same module. Both are keyed by .id (= toId(itemId)) in declaredThings/declaredProtos, so if the two id producers (the loader's symCounter and cg's idgen) ran independently they could mint the same item: e.g. a rttiDestroy wrapper and the very =destroy hook it wraps both land on backend item 21 -> one masks the other in declaredThings -> the hook's body is never emitted -> "undefined reference" at link. Drawing every backend sym from this one counter keeps them disjoint. Returns -1 if the module is not loaded yet (then the caller falls back to the idgen's own counter — only reachable for sem-time @bk minting, whose module is never loaded in that process). Source   Edit   
proc nextBackendTypeItem(c: var DecodeContext; module: int32): int32 {.
    ...raises: [KeyError], tags: [], forbids: [].}
TYPE analogue of nextBackendSymItem: the lower/cg stages mint fresh backend TYPES (closure-env objects, ptr wrappers) whose itemId must not collide with the module's loaded types. Drawn from the per-module typeCounter, which moduleId seeds from the file's (unusedid) so the first minted type sits ABOVE every frontend/lowered type item. Returns -1 if the module is not loaded (caller falls back to the idgen's own counter). Source   Edit   
proc nifModuleHasIncludes(config: ConfigRef; fileIdx: FileIndex): bool {.
    ...raises: [OSError, IOError],
    tags: [RootEffect, ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}
Cheap header-only check: does the module's <suffix>.nif contain an (include ...) record? Used by nimsuggest (moduleFromNifFile) to refuse to NIF-serve modules that include files, so the includer is source-compiled and the included symbols never round-trip through NIF (which mishandles their owner/line-info on reload). Source   Edit   
proc parseSymName(s: string): ParsedSymName {....raises: [], tags: [], forbids: [].}
Source   Edit   
proc registerNifAstTags() {....raises: [], tags: [], forbids: [].}
(Re)registers ast2nif's NIF tags explicitly. The top-level registerTag initializers above depend on nifstreams.pool having been initialized FIRST (pool = createLiterals(TagData) in nifstreams' module init) — an inter-module init-order requirement. The IC-built compiler currently emits module init calls in a different order, so the initializers registered into a pool that was subsequently replaced: the tag ids then denoted builtin tags (replay came out as deref, repdestroy as pat, ...) and every written NIF was silently corrupted. Called from nim.nim before any command runs; idempotent (getOrIncl by name). Source   Edit   
proc resolveHookSym(c: var DecodeContext; name: string): PSym {.
    ...raises: [OSError, IOError, KeyError],
    tags: [ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}
Source   Edit   
proc scanIncludeGraph(config: ConfigRef): seq[
    tuple[includer: string, includes: seq[string]]] {.
    ...raises: [OSError, IOError],
    tags: [ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}

Standalone "full table" scan of every <suffix>.nif in the nimcache: reads only each module's header records — (modulesrc "path") (the includer's own source) and (include . . "path"...) (resolved included files) — and returns (includerSource, includedSources) pairs for the modules that include anything. No DecodeContext, no symbol/index loading: it parses the few dep tokens at the top of the file and stops at the first non-dep node.

Used by nimsuggest to answer, for a cold-opened include file, "which module includes me?" without NIF-loading that module — so the includer can be source-compiled (modules that include files are never served from NIF).

Source   Edit   
proc sealLoadedRoutines(c: var DecodeContext) {....raises: [], tags: [],
    forbids: [].}
Before writeLoweredModule re-serializes the lowered module, seal ONLY the module's ROUTINE syms. A .t.nif written by writeLoweredModule is the SOLE source the cg stage loads (there is no .s.nif fallback for its bodies), so every type, global, param and local must still emit a REAL def in it — only cross-routine references may be SymUses (each routine's def is emitted once, at module scope, by the explicit stub loop). Types/globals stay Complete so writeType/writeGlobals emit them; routines become Sealed so a body referencing another routine writes a SymUse resolved through the module index. Source   Edit   
proc setMainModule(c: var DecodeContext; fileIdx: FileIndex) {....raises: [],
    tags: [RootEffect], forbids: [].}
Records the module that is being compiled fresh so that re-exports of its own symbols by dependencies are not turned into duplicate stubs. Source   Edit   
proc toClassSymId(config: ConfigRef; typeId: ItemId): nifstreams.SymId {.
    ...raises: [], tags: [RootEffect], forbids: [].}
Converts a type ItemId to its SymId for the class index. Source   Edit   
proc toConverterIndexEntry(config: ConfigRef; converterSym: PSym): (
    nifstreams.SymId, nifstreams.SymId) {....raises: [], tags: [RootEffect],
    forbids: [].}
Converts a converter symbol to an index entry (destType, converterSym). Returns the destination type's SymId and the converter's SymId. Source   Edit   
proc toHookIndexEntry(config: ConfigRef; typeId: ItemId; hookSym: PSym): HookIndexEntry {.
    ...raises: [], tags: [RootEffect], forbids: [].}
Converts a type ItemId and hook symbol to a HookIndexEntry for the NIF index. Source   Edit   
proc toMethodIndexEntry(config: ConfigRef; methodSym: PSym; signature: string): (
    nifstreams.SymId, nifstreams.StrId) {....raises: [], tags: [RootEffect],
    forbids: [].}
Converts a method symbol/signature to a method index entry. Source   Edit   
proc toNifFilename(conf: ConfigRef; f: FileIndex): string {....raises: [OSError],
    tags: [RootEffect, ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}
Source   Edit   
proc tryResolveCompilerProc(c: var DecodeContext; name: string;
                            moduleFileIdx: FileIndex): PSym {.
    ...raises: [OSError, IOError, KeyError, Exception, ValueError],
    tags: [RootEffect, ReadEnvEffect, ReadIOEffect, ReadDirEffect], forbids: [].}
Tries to resolve a compiler proc from a module by checking the NIF index. Returns nil if the symbol doesn't exist. The NIF disamb is mint order, so name.0. can be any of the overloads sharing the name — for newSeq it is the generic magic, not the RTL proc (a refc build then demands codegen of the generic and dies on seq[T]): enumerate the index entries with this basename and pick the one that carries sfCompilerProc. Source   Edit   
proc writeLoweredModule(c: var DecodeContext; config: ConfigRef;
                        precomp: PrecompiledModule; hooks: openArray[LogEntry];
                        outfile: string) {.
    ...raises: [KeyError, Exception, OSError, IOError], tags: [RootEffect,
    ReadEnvEffect, ReadIOEffect, ReadDirEffect, WriteIOEffect], forbids: [].}
Re-serialize a backend-loaded module as a FULL module NIF (.t.nif) whose routine (sd) entries carry their TRANSFORMED bodies (the lower stage set them, recursively lifting nested closures — including the async state-machine procs whose inner closure the per-(lowered)-entry path failed to cross) and whose lambda-lift-minted entities (closure-env types/syms, lifted nested procs) are real, indexed defs. The cg stage then loads it through the normal module loader (moduleFromNifFile), so a transformed body arrives via loadSymFromCursor's Step-A 2-way-body slot WITH the lifted signature — no (lowered) side-car, no :envP re-weld. This realizes ic_ideas.md's eager two-way body whole-module. Source   Edit   
proc writeNifModule(config: ConfigRef; thisModule: int32; n: PNode;
                    opsLog: seq[LogEntry]; replayActions: seq[PNode] = @[];
                    implDeps: seq[int] = @[];
                    reexportedModules: seq[(string, string)] = @[];
    genericOffers: seq[tuple[generic, inst: PSym, concreteTypes: seq[PType],
                             genericParamsCount: int]] = @[];
                    typeOffers: seq[tuple[generic: PSym, inst: PType]] = @[];
                    resolvedImportDeps: seq[FileIndex] = @[];
                    firstUnusedId: int32 = 0) {.
    ...raises: [KeyError, Exception, OSError, IOError, ValueError], tags: [
    RootEffect, ReadEnvEffect, ReadIOEffect, WriteDirEffect, ReadDirEffect,
    WriteIOEffect], forbids: [].}
Source   Edit   
proc writeSemDeps(config: ConfigRef; thisModule: int32; importPaths: seq[string]) {.
    ...raises: [OSError, IOError, Exception], tags: [RootEffect, ReadEnvEffect,
    ReadIOEffect, WriteIOEffect, WriteDirEffect, ReadDirEffect], forbids: [].}

The module's REAL direct imports as nim m sem resolved them — static plus any a macro generated — recorded as full source paths. nim ic reads this .s.deps.nif to re-derive the build graph: imports the static scanner missed become new nodes (replacing the old build-failure discovery loop), and when false imports the scanner over-included are pruned. Always written so it is current after every successful sem (like .edges).

Ported to nifcore: delegates to icnifcore.writeSemDeps (Stage 1 of the NIF-stack migration; see doc/ic_nifcore_port.md). Output is byte-identical to the previous nifstreams writer.

Source   Edit  

Made with Nim. Generated: 2026-07-02 04:38:16 UTC