dcrubro/skalacoin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: dcrubro/skalacoin:txtest
Branches Tags
dcrubro/skalacoin:master dcrubro/skalacoin:nodediscovery dcrubro/skalacoin:txtest dcrubro/skalacoin:p2ptest
..
compare: dcrubro/skalacoin:a22234ce90a35951b23fc2c4e2f87d7fb85b6eea
Branches Tags
dcrubro/skalacoin:master dcrubro/skalacoin:nodediscovery dcrubro/skalacoin:txtest dcrubro/skalacoin:p2ptest
28 Commits
txtest .. a22234ce90
Author SHA1 Message Date
dcrubro a22234ce90 update .gitignore 2026-09-16 15:33:36 +02:00
dcrubro f2d0579a57 fix warnings 2026-09-15 18:07:33 +02:00
dcrubro fd27c2bac4 Rename extern/global variables to g_varName instead of varName to avoid shadowing 2026-09-15 17:54:50 +02:00
dcrubro 099433b9df Remove the hand-rolled ad-hoc DynArr and DynSet, replace with dlibc vector_t and set_t 2026-09-15 14:30:29 +02:00
dcrubro f40ffaa6f7 verbosity 2026-08-16 23:23:32 +02:00
dcrubro d7bcd64130 Fix analyzer conflicts 2026-08-16 23:15:57 +02:00
dcrubro af888258f4 cmake analyzer and strict modes 2026-08-16 22:08:47 +02:00
dcrubro 914fa6e5a7 Change cmakelists for compile strictness 2026-08-16 20:48:24 +02:00
dcrubro 309367a91e Add sendrawtx command, specify send command to not broadcast 2026-08-03 17:28:12 +02:00
dcrubro 10d5d71a9f Prevent transaction replay and stop reorgs from destroying transactions 
Two gaps in the account model, found while working out how many confirmations
are actually needed for a payment to be safe.

== A reorg silently destroyed transactions ==

chain.c removed transactions from the mempool when they were mined but never
put them back. TxMempool_Insert had exactly two callers -- the `send` command
and inbound broadcasts -- so a transaction that existed only in an orphaned
block was gone from both the chain and the pool, and would never be mined
unless its sender happened to rebroadcast.

That makes the usual reassurance about accidental orphans ("your transaction
just lands in the next block") false for this node. Chain_RollbackToHeightLocked
now returns the discarded blocks' non-coinbase transactions to the mempool
before anything is freed, going through the existing Chain_BorrowBlockTransactions
disk fallback because those blocks are usually header-only by then.

Coinbases are deliberately not restored: they are bound to a specific height and
reward and are invalid anywhere else.

A transaction that also appears in the replacing branch needs no special
handling. Chain_ReplaceBranch rolls back and applies under a single lock
acquisition, and Chain_AddBlockLocked removes every applied transaction from the
pool again -- so it is re-inserted and removed moments later, with no window in
which a miner could pick up a copy of something already back in the chain. That
ordering is what makes this safe in an account model, where re-mining a
transaction would debit the sender twice, so it is spelled out at the site.

== Any historical transaction could be replayed ==

Nothing rejected a transaction whose hash was already in the chain.
Block_AllTransactionsValid checks signatures and that there is exactly one
coinbase; it never looked for repeats, and the mempool's hash-keyed dedup is
mempool-only. So anyone could take a mined, publicly visible signed transaction,
rebroadcast it, and have it mined again -- debiting the sender a second time.
UTXO chains get this for free because the spent inputs no longer exist; an
account model has nothing to stop it.

balance_sheet_entry_t gains lastTxTimestamp, and a non-coinbase transaction is
now valid only if its timestamp is strictly greater than its sender's last
included one. Transaction timestamps are unix MILLISECONDS (the comment in
transaction.h claimed seconds and was stale), so two genuinely distinct
transactions never collide and an exact collision means a byte-identical copy --
precisely what must be refused.

Coinbase is exempt: a block may hold only one, its amount is pinned to the
height, and including someone else's would only donate the miner's own reward.

Three deliberate choices:

  * Enforced in Chain_AddBlockLocked, BEFORE the push. That function is the only
    insertion point into the chain besides the header-only disk load, so mining,
    broadcast, windowed sync, orphan attach and reorg apply are all covered by
    one check rather than four copies. Putting it in the ledger pass instead
    would be too late -- that runs after the block is in the chain and can only
    return false, leaving an invalid block behind.
  * Extracted as Chain_BlockRespectsSenderOrdering rather than left inline,
    so the multi-sender case can be tested directly. Senders are strictly
    independent: one account's timestamps say nothing about another's, and
    folding them together would reject ordinary blocks outright.
  * DebitAddress takes the timestamp and advances the guard in the same call, so
    a spend cannot happen without the guard moving. They cannot drift apart.

Rebuilt for free on reorg: the rollback already destroys and replays the whole
balance sheet, so setting lastTxTimestamp in that same loop means there is no
separate invalidation path to get wrong.

== The miner's fee sort broke the new rule ==

CompareTransactionPriority orders by fee descending, so a sender's later,
higher-fee transaction could sort ahead of their earlier, lower-fee one -- and
Chain_AddBlockLocked walks a block in order, so the node would have built blocks
its own rule rejects.

This cannot be folded into the comparator: "higher fee first, except same sender
by time" is not a strict weak ordering (A beats B on fee, B beats C on fee, C
beats A on time) and qsort with an inconsistent comparator is undefined. Instead
the priority sort is followed by a permutation restricted to each sender's own
slots, so fee-based slot allocation survives untouched and only the order within
one sender's slots changes.

== Mempool timestamp policy (local policy, NOT consensus) ==

Separate concern: keeping junk out of the pool. TX_MAX_FUTURE_DRIFT_MS (2h) and
TX_EXPIRY_MS (4 days, chosen to roughly match what DIFFICULTY_ADJUSTMENT_INTERVAL
spans but in milliseconds so it does not drift with block time), both in
constants.h. Gated at both admission sites via TxMempool_PolicyAccepts, with
TxMempool_PruneExpired on the 1Hz maintenance tick.

Blocks are never rejected for either bound, so a node with a skewed clock cannot
fork itself off the network over an admission rule.

Both bounds are measured against the node's own clock, NOT against the chain
tip. Measuring "future" against the last block assumes blocks keep arriving: on a
quiet chain the tip can be hours old, and an honest transaction created right now
would look hours ahead of it and be refused -- making sending impossible exactly
when the chain is idle.

A too-old timestamp needs no rule here; the replay guard already refuses anything
at or below a sender's last.

== Verification ==

New unit suite, 21 assertions against the real objects, all passing. The ones
that matter:

  multi-sender independence
    alice(6000) and bob(101) in one block          -> accepted
    reversed order                                  -> accepted
    8 senders sharing one timestamp                 -> accepted
  replay
    equal to sender's last (carbon copy)            -> rejected
    older than sender's last                        -> rejected
    same transaction twice in one block             -> rejected
  ordering within a block
    same sender increasing                          -> accepted
    same sender decreasing                          -> rejected
  policy window
    inside/outside both bounds, and pruning         -> as specified

Without the sender comparison in the guard, the first three would all fail --
that is the case worth guarding against, because the check reads as if it folds
all senders together.

penalty_test and dag_test suites still pass. AddressSanitizer reported zero
errors on both nodes across the reorg path, which is the signal that matters
given the rollback now does more work.

== Not yet verified ==

End-to-end replay rejection and reorg-restores-mempool against live nodes; both
need two funded wallets, so they are a separate setup. The fork regression
(forksync 5 60) was still climbing toward its penalty threshold when this was
written -- branchLead 40 of the required 42, behaving correctly but not yet
adopted.

== Note ==

balance_sheet_entry_t is written raw to disk, so balance_sheet.data gains a
field and old files will not load. wipechain before running.

Related gap, pre-existing and NOT addressed: the disk load restores headers only
and Chain_RecomputeRuntimeState does not rebuild balances, so lastTxTimestamp
survives a restart purely because it is persisted in balance_sheet.data -- and
that file has no height marker to detect it being stale against the chain. A node
with a missing or out-of-date sheet silently resets every account's guard to 0
and allows one replay per account. Balances are already wrong in that situation
today, but this change turns it from an accounting bug into a security one.
Recording the chain height alongside the sheet and refusing to start on a
mismatch is the natural follow-up.
 2026-08-03 16:50:16 +02:00
dcrubro 393d26dfcb Fix the difficulty adjustment off-by-one; slight initial diff adjustment; BREAKS CONSENSUS 2026-08-03 15:45:53 +02:00
dcrubro 5eaf0b699c Let a node that is behind adopt the peer's branch, and fix two bugs that only appear once a reorg actually applies 
Reported: a node at height N+10 on its own short fork, with the peer at N+500,
could never sync. It located the fork point correctly, pooled the branch, and
then refused it forever. At 90s blocks a node is 960 blocks behind after a day
offline, so this is the normal case, not an edge case.

== Why it could never recover ==

The reorg delay was served by LOCAL chain growth alone:

    elapsed = tipHeight - observedTip

That only makes sense for a node whose tip is advancing. A node that is behind
has a frozen tip precisely because it is rejecting the branch, so elapsed stays
0 forever while penalty(5) = 42. Both escape hatches also fail: mining out of it
means extending a fork nobody accepts, and the IBD exemption never fires for a
node that is mining, because mining keeps its tip fresh.

Being 490 blocks behind is being behind, not a reorg contest. Refusing to adopt
protects nothing while the node falls a further 40 blocks behind every hour.

The delay is now satisfiable by EITHER side making progress:

    localGrowth = tipHeight - observedTip
    branchLead  = candidateTip - tipHeight        (0 if not ahead)
    elapsed     = max(localGrowth, branchLead)

A branch already extending `penalty` blocks past our tip has demonstrated
exactly what the delay asks for, and every one of those blocks carries PoW we
validated ourselves. Requiring us to independently produce the same amount is
demanding the same proof twice. Only VALIDATED blocks count -- a peer's
advertised height is not evidence and never reaches this code.

It degrades correctly in both directions: a mining node's tip advances, so an
attacker must outpace it by penalty blocks; a node that is only observing
follows the heaviest chain, which is what an observer should do.

== The window has to keep pulling the branch ==

Every forked delivery reset nextReq back to our own tip, so the window
re-requested the same eight heights forever and the pool never grew past the
window size -- branchLead could not rise even in principle. The backward walk
now runs ONCE to establish linkage, then the window marches forward pooling the
branch, retrying adoption per window-full with a final attempt when it drains.

== sync force ==

Operator override that skips the delay for one sync, for a node whose chain is
known to be the wrong one. Threaded explicitly (Chain_ReplaceBranch gains
bypassPenalty, OrphanPool_AttemptAttachForced) rather than through a global, so
nothing a peer sends can reach it. Linkage, work comparison and atomicity still
apply -- it waives only the waiting, and says so loudly in the log.

== Bug found in testing: PoW is branch-relative ==

A block's epoch seed is the last block of the previous epoch ON ITS OWN BRANCH.
Validating a competing branch's block against OUR epoch seed does not merely
fail to resolve: when the chains diverge before the boundary it resolves to the
WRONG seed and rejects a perfectly valid block. Any fork spanning an epoch
boundary was therefore impossible to assemble -- the branch could not grow past
the boundary block, so branchLead stalled one short of it.

The receive path now does self-contained checks only (Block_HasValidStructure:
merkle, transactions, vote, non-empty). Proof of work moved to
Chain_AddBlockLocked, at the point a block joins the chain, where the branch
context is real -- the rollback has put its ancestors in place by then. That is
where the invariant belongs and it removes a duplicate check rather than adding
one. Needs Chain_DagParamsForHeightLocked, because Chain_AddBlockLocked already
holds chainLock for writing and the lock is not recursive.

Consequence worth knowing: the orphan pool can now hold blocks whose work has
not been verified, bounded by MAX_ORPHAN_BLOCKS (512). Each still had to pass
merkle and full transaction/signature validation, and none can reach the chain
unverified.

This bug also affected plain forward sync across block 350000; it was masked
because appending keeps the boundary block in the chain.

== Bug found in testing: stale DAG accepted as current ==

Block_PowHashHeavy matched on epoch index and size but not the seed. A DAG's
content is a function of (seed, size); the epoch index is a label for it. A
reorg is exactly the event that changes the seed while leaving index and size
untouched, so mid-apply the miner's context still held a DAG built from the
PRE-reorg seed, the guard passed, and a valid block was hashed against the wrong
lanes. g_dagSeed now records what each DAG was generated from and both
Block_EnsureAutolykos2Dag and Block_PowHashHeavy compare it.

== Bug found in testing: double free on the failed-apply path ==

SIGABRT in the allocator: free_tiny_botch -> DynArr_destroy -> Block_Destroy ->
Chain_FreeBlockArray -> Chain_ReplaceBranch.

DynArr_push_back stores the struct BY VALUE, so the chain's element and the
caller's block_t share one transactions pointer. Three places free that array
through the chain's copy -- Chain_ClearBlocks, Chain_RollbackToHeightLocked and
Chain_SaveToFile -- and each NULLs only the chain's side, leaving any caller
wrapper dangling. Whether a caller then had to use free() or Block_Destroy() was
a convention carried in comments at every call site plus a consumed-count passed
into Chain_FreeBlockArray. Chain_ReplaceBranch reset that count to 0 after
rolling back a failed apply, which told the cleanup to Block_Destroy exactly the
blocks whose arrays the rollback had just freed.

Rather than fix the count, the aliasing is now safe by construction:
Chain_AddBlockLocked clears the CALLER's transactions pointer immediately after
the push. Since DynArr_destroy(NULL) is a no-op, free(wrapper) and
Block_Destroy(wrapper) become equivalent and both safe regardless of what later
frees the chain's copy. The consumed-count parameter and both counters are gone
-- the thing that could be got wrong no longer exists -- and all call sites are
unified on Block_Destroy.

Placement is deliberate: immediately after the push, not at the end on success.
The ledger pass can fail with the block already in the chain, returning false to
a caller that destroys its wrapper on failure -- OrphanPool_ExtendTip does
exactly that, a third live instance not yet triggered.

Two follow-ons the refactor forced, both improvements anyway: MineAndAppendBlock
read the coinbase for its log line after the add (hoisted above it), and the
success log printed the caller's block rather than the chain's copy.

== Also ==

The deferral line is rate-limited. The maintenance thread retries pooled
branches once a second and elapsed only changes when something moves, so it
printed an identical line every second -- forever, on a node that is not mining.
It now reports each distinct situation once.

== Verification ==

Synthetic fork, node A 5 deep, node B ~60 ahead, EPOCH_LENGTH=8 so the branch
crosses three epoch boundaries:

  branchLead climbs 8 -> 16 -> 28 -> 34, crosses penalty(5)=42
  Adopted competing branch of 50 block(s) at fork height 20
  sync complete: localHeight=70
  Chain OK

Repeated under AddressSanitizer: adopted 47 blocks, 0 ASan errors on both nodes.
This matters because the refactor rewrites the exact cleanup path the SIGABRT
came from, and a double free that no longer aborts would otherwise pass silently.

Unit suites pass, including a new assertion "heavy path refuses a DAG built from
a different seed" -- the direct regression for the stale-DAG bug.

Harness note: each node needs its OWN wallet. With a shared one both pay the
same coinbase address, produce identical merkle roots, and at easy difficulty
mine byte-identical blocks -- the fork test silently became a catch-up test.

== Still untested ==

The restore-after-failed-apply path is no longer naturally reachable now that
the two bugs above are fixed, so it needs deliberate corruption to exercise.
Test B (branch only slightly ahead must still DEFER), test C (sync force), and a
TSan pass over the changed paths are outstanding.
 2026-08-02 19:36:09 +02:00
dcrubro 01c44731ef Fix windowed sync stalling against a peer on a competing branch 
Reported symptom: node A (4656 blocks, last 10 mined on its own fork) syncing
from node B (5646 blocks) crawled for minutes while B re-served the whole chain
several times over. A's log was thousands of "Queued orphan BLOCK_DATA" lines
interleaved with "timed out fetching block N, giving up", sliding forward 8
blocks at a time and never reorging.

== Root cause ==

The sync loop had no channel for FETCH_BLOCK replies. It inferred that a
requested block had arrived purely from the chain growing:

    if ((uint64_t)Chain_Size(chain) > h) { /* block h was applied */ }

But the reply is handled on the peer's io thread by Node_ParseAndAcceptBlock,
which correctly identifies a block whose prevHash points at the peer's branch
and files it in the orphan pool -- the chain does not grow. So a block that
ARRIVED AND WAS CORRECTLY ORPHANED was indistinguishable from a lost packet.

Every one of the 990 blocks therefore cost MAX_SYNC_RETRIES re-requests plus a
SYNC_REQUEST_TIMEOUT_MS stall before the window slid on and did it again. That
is both the crawl and the repeated serving on the peer.

Neither existing fork probe could rescue it:

  * The divergence check sits downstream of a SUCCESSFUL append -- it only runs
    after Chain_GetBlockCopy(chain, h) succeeds. When the fork begins at the
    very first requested height nothing ever lands, so RequestForkWindow was
    never reached from there.
  * The no-progress fallback only runs after the entire window range drains,
    i.e. after timing out through all 990 blocks, and is capped at 3 rounds.

== Delivery receipts ==

Node_NoteBlockDelivered / Node_TakeBlockDelivery / Node_ResetBlockDeliveries,
backed by a 512-slot ring with its own mutex. Recorded from the BLOCK_DATA
handler only -- BLOCK_DATA is sent solely in reply to a FETCH_BLOCK, so a
receipt means "the peer answered", independently of whether the block could
join our chain. A receipt for a height already present is refreshed in place,
so a retried request cannot leave a stale one for the next window to consume.

Receipts carry a status rather than a bool:

    NODE_DELIVERY_APPENDED   joined our chain
    NODE_DELIVERY_DUPLICATE  we already held this exact block -- common ground
    NODE_DELIVERY_ORPHANED   competing branch; now pooled
    NODE_DELIVERY_REJECTED   failed validation

DUPLICATE is what lets a backwards fork walk terminate; a plain "appended"
boolean cannot tell "we already have this" from "this is on their branch",
which is precisely the signal the walk needs.

The sync loop now treats any non-APPENDED delivery as the divergence signal and
probes immediately, instead of retrying something retrying cannot fix. Bounded
by MAX_FORK_PROBE_ROUNDS -- without that the window resets to the same height
and re-probes forever -- and the counter resets on real progress so a branch
that forks again further on can still be followed.

== RequestForkWindow is now an actual walk ==

It previously fired REORG_FETCH_DEPTH (128) requests and slept a flat
SYNC_REQUEST_TIMEOUT_MS. Two problems:

  * The sleep was a race against the peer's serving rate. At the ~10 blocks/s
    observed in testing, a 128-block window cannot land in 5s, so the
    OrphanPool_AttemptAttach that followed ran against a half-filled pool and
    reported a failure that was not real.
  * It always asked for the full depth. On a shallow fork the overwhelming
    majority came back as duplicates that were discarded without even entering
    the pool -- a wasted full block send each.

It now descends in batches of MAX_PARALLEL_FETCHES, waits on that batch's
receipts rather than guessing, and stops at the first height the peer returns
that we already hold: that block is the fork point and everything below it is
shared. It returns whether common ground was found, so callers no longer
attempt an attach that cannot possibly link.

Measured on the reported scenario: 16 requests instead of 129, locating the
fork point at height 4645 (fork = blocks 4646..4655, exactly the 10 mined).

== IBD_TIP_AGE_BLOCKS 500 -> 20 ==

The reorg penalty is served by LOCAL chain growth, so a node that does not mine
can never serve it -- its tip does not move, and the only way it could grow is
by adopting the branch the penalty is gating. The IBD exemption is that node's
only route back, so at 500 block times it had to sit stalled for ~12.5 hours.
20 block times is ~30 minutes at a 90s target, far beyond normal Poisson block
spacing (a gap that long has probability ~e^-20), so a node genuinely following
the tip will not trip it.

This is what flipped the live test from "initialSync=no" to "initialSync=yes"
and let the reorg be adopted at all. Noted in the constant's comment: the
exemption is all-or-nothing, so lowering it further widens that hole.

== Reorg failures are now legible ==

Chain_ReplaceBranch broke silently for fork-point-beyond-tip, work computation
failure, insufficient work, snapshot failure and candidate copy failure alike,
so "not adopted" could not be diagnosed from a log. Each now says which. The
not-heavier case reports candidate vs incumbent block counts, because the
common cause is a branch that is still arriving -- a partial branch is
genuinely lighter -- and that reads very differently from a peer actually on a
weaker chain.

The sync loop's "Reorg candidate not adopted (lighter branch, or still serving
its reorg penalty)" is replaced by "Reorg not completed on this pass; branch
stays pooled for retry". The old wording was actively misleading: live testing
showed it firing on reorgs that the 1Hz maintenance thread completed a second
later.

== Verification ==

Against a real peer at 5646 blocks, from a real local chain of 4656 with a
10-block fork:

                        reported    receipts only   all changes
  wall-clock            minutes     123s            116s
  timed-out fetches     1 per block 0               0
  fork-walk requests    n/a         129             16
  reorg completed       never       maintenance     first pass
  final height          4656        5646            5646
  fullverify            -           Chain OK        Chain OK

The 116s is not a meaningful speedup over 123s and should not be read as one:
990 blocks at ~10 blocks/s is ~100s, so both are bound by the peer's serving
throughput rather than by anything in the sync loop. The substantive wins are
the 129 -> 16 fork-walk reduction, the reorg completing deterministically
instead of by luck, and logs that no longer report failure on success.

Also exercised: a synthetic partitioned fork (10 vs 60 blocks) confirming zero
timeouts and bounded probe rounds when the branch is correctly refused by the
penalty.

== Known limits, unchanged ==

Chain_ReplaceBranch adopts whatever is pooled at attach time (18 blocks in the
live run); the remainder arrives through normal windowed sync afterwards. That
bounds a single reorg by MAX_ORPHAN_BLOCKS.
 2026-07-30 21:51:09 +02:00
dcrubro 0e721ca389 Fix reorg penalty scaling direction and rebuild DAG sizing around a miner-signalled band 
Two consensus-parameter bugs, both latent today and both guaranteed to become
live splits later. Neither is a regression; both are cheap to fix now and
expensive to fix after launch.

== 1. The reorg penalty scaled with block time the wrong way ==

FetchScheduler_ComputeReorgPenaltyBlocks had TARGET_BLOCK_TIME in the numerator
and REORG_PENALTY_REF_BLOCK_TIME in the denominator. The penalty is counted in
BLOCKS, so the wall-clock protection it actually buys was

    penalty(d) * TARGET_BLOCK_TIME  ~  d^2 * T^2 / REF

i.e. quadratic in block time, when the stated intent in constants.h is
wall-clock equivalence to the reference scheme defined at REF_BLOCK_TIME = 150.
At T = 90 the chain got 54*d^2 seconds of protection instead of 150*d^2 --
2.78x weaker than intended -- and any future block-time reduction would have
weakened it further, silently.

Swapping the two makes T cancel out of the wall-clock figure:

    penalty(d) = ceil(d^2 * REF / T)   ->   protection ~ d^2 * REF seconds

Penalty in blocks, depth -> before/after: 4 -> 10/27, 8 -> 39/107,
10 -> 60/167, 50 -> 1500/4167, 100 -> 6000/16667.

Verified by rebuilding at TARGET_BLOCK_TIME = 60: wall-clock protection is now
identical at both block times (2400s, 9600s, 15000s, ...), ratio 1.0000. Under
the old formula 60s blocks would have been 56% weaker.

Still integer-only: the ceiling division and saturation guards are unchanged,
and numeratorScale only moves 90 -> 150 while `raised` stays bounded by
REORG_PENALTY_MAX_DEPTH^2 = 1e6, nowhere near overflow.

This is fork choice, not block validity, so mixed-version nodes converge once
the longer penalty expires and no block ever becomes invalid.

== 2. DAG sizing was a fail-open consensus split waiting on block 350000 ==

Fixed, in rough order of severity:

  * FAIL-OPEN PoW. Block_CalculateAutolykos2Hash memset the hash to zero on any
    failure, and zero compares below every target -- so a DAG that was missing,
    mis-sized or had failed to build made *every* block pass PoW validation
    instead of rejecting it. PoW checks now fail closed.

  * TWO DIVERGENT IMPLEMENTATIONS of the same rule: CalculateTargetDAGSize in
    constants.h (mining) and ComputeEpochDagBytesForHeightFromChain in main.c
    (verification). They already disagreed at exactly height == EPOCH_LENGTH,
    where the constants.h copy underflowed size_t computing
    `Chain_Size - 1 - EPOCH_LENGTH` and returned 0.

  * A MEANINGLESS DIFFICULTY COUPLING. difficultyTarget is compact-encoded
    [1B exponent][3B coefficient]; subtracting two compact values mixes
    exponent and mantissa. The first real retarget yields a delta of 900649,
    which times DAG_BASE_GROWTH is ~967 TB, so the result was always clamped
    and the "proportional" term degenerated to a binary base+maxUp/base-maxDown
    switch. The size also never accumulated -- it was always recomputed from
    the constant base -- so the documented 1 GB/epoch growth could not happen.

  * A NON-EPOCH-ALIGNED SEED. GetNextDAGSeed returned hash(tip) while the
    verifier correctly used hash(block[epochIndex * EPOCH_LENGTH - 1]). The
    `chainSize % EPOCH_LENGTH == 0` rebuild guard masked this while running,
    but startup called it at an arbitrary height -- so a node restarted
    mid-epoch built a DAG from a different seed than one that had run straight
    through the boundary, and its blocks were rejected.

  * UNSIGNED PROMOTION. `DAG_BASE_GROWTH * difficultyDelta` promoted the signed
    delta to unsigned long long and wrapped before being assigned back to
    int64_t; the main.c copy cast first, so the two also differed in overflow
    behaviour. Both are gone.

  * A LEAK on the `targetSize <= 0` path in main.c (both block copies).

  * A USE-AFTER-FREE at every epoch boundary: Block_RebuildAutolykos2Dag ran
    DagClear -> DagAllocate -> DagGenerate from whichever thread advanced the
    tip, freeing ctx->dag.buf while miners read it.

  * GetAutolykos2Ctx called DagAllocate without DagGenerate, leaving
    dag.len == 0 so every heavy hash failed -- which, combined with the
    fail-open above, meant "everything is valid". It also memset 1 GiB that
    DagGenerate immediately overwrote.

  * TOCTOU / lock reentrancy: CalculateTargetDAGSize called Chain_Size 4x and
    Chain_GetBlockCopy 2x, each taking chainLock for reading, making it unsafe
    to call from a write-locked section.

--- New sizing rule: default-grow inside a hard band ---

DAG size now follows a recurrence gated by a miner signal in the block header,
clamped to [DAG_MIN_SIZE, DAG_MAX_SIZE]:

    brake  = (hold + down) * DAG_BRAKE_DEN > EPOCH_LENGTH * DAG_BRAKE_NUM
    downQ  =          down * DAG_DOWN_DEN  > EPOCH_LENGTH * DAG_DOWN_NUM

    downQ(k) && downQ(k-1) -> size -= DAG_EPOCH_STEP   (floored at DAG_MIN_SIZE)
    brake                  -> size unchanged
    otherwise              -> size += DAG_EPOCH_STEP   (capped at DAG_MAX_SIZE)

Growth is the default and there is deliberately NO up-vote: every signal a
miner can express only slows the walk or reverses it. Under stratum-style
pooled mining the pool builds the header and therefore controls its share of
the vote, so the mechanism has to be safe under pool capture -- and it is,
because the lever a pool would want (grow the DAG to price out smaller miners)
does not exist. This is NOT because upward capture would be self-defeating: it
would in fact be profitable, since the fixed block reward redistributes to
whoever survives and difficulty retargets down. The protection is the absence
of the lever. The whole upward trajectory is therefore governance
(DAG_EPOCH_STEP, DAG_MAX_SIZE), not signalling.

Braking keeps the DAG small, which helps old hardware and only costs ASIC
resistance -- bounded by DAG_MIN_SIZE, which is the constant that actually
secures the property. Shrinking needs a supermajority sustained across two
consecutive epochs; that gates the onset only, so miners genuinely being
squeezed get relief every epoch rather than every other one.

Thresholds are cross-multiplied rather than divided, so there is no rounding
for nodes to disagree on, and the denominator is the constant epoch length
rather than blocks-observed, so a partial epoch cannot read as a stronger
signal than it is. No floating point anywhere on this path.

--- Header vote field ---

reserved[0] carries the vote: 0 = grow (default), 1 = hold, 2 = down.
reserved[1..2] must be zero. All three already sat inside the packed, hashed
header, so the vote is committed to by both the canonical hash and the PoW hash
and cannot be altered after mining -- no wire-format or hash-layout change.

0 must mean grow, because the point of this shape is that inaction produces
growth; it also means a miner that knows nothing about the vote contributes to
the intended default rather than silently freezing the schedule.

Rejecting unrecognised vote values and non-zero spare bytes is a new validity
rule. It closes 24 bits of undefined-meaning malleable header space.

--- Validation moves to the light path ---

Autolykos2_DagGenerate fills lane i with exactly what ReadDagLaneFromSeed
recomputes for lane i -- Blake2b(seed || (i/2)_LE64), half i&1 -- so the DAG is
a pure cache and the two hashing paths are bit-for-bit equivalent. Validation
therefore uses the light path: no allocation, correct for any epoch rather than
only whichever one the global DAG happens to hold, and the DAG band becomes a
miner requirement rather than a full-node memory requirement.

Chain_OnTipAdvanced no longer rebuilds the DAG at all. MineBlock builds it on
demand for the height it is working on, so a node that does not mine never
allocates one, generation stays off the tip-advance path, and the buffer is
only ever touched by the miner (the ctx mutex remains as a backstop).

--- API changes ---

  + Chain_DagParamsForHeight(chain, height, &dagBytes, seed) -- single source
    of truth for both the size and the epoch seed, backed by a memoised
    per-epoch table on blockchain_t. The table is a pure cache of a function of
    the headers, extended lazily and dropped whenever anything at or below the
    tip changes; it lives on the chain rather than in a global because a
    second, header-only chain is built to re-verify historical PoW. Guarded by
    a per-chain mutex, always taken after chainLock.
  + Block_EnsureAutolykos2Dag / Block_PowHashHeavy / Block_PowHashLight.
    The heavy variant verifies the epoch and size itself, so it can never
    answer from a DAG built for another epoch.
  + Block_HasValidProofOfWorkWithParams -- resolve-once form. MineBlock called
    Block_HasValidProofOfWork inside its nonce loop, so resolving from the
    chain per attempt would have taken chainLock millions of times per block.
  + Block_HasValidVote.
  ~ Block_HasValidProofOfWork / Block_IsFullyValid now take the chain, because
    PoW validity genuinely is chain-relative. blockchain_t gained a struct tag
    so block.h can forward-declare it.
  - CalculateTargetDAGSize, GetNextDAGSeed, ComputeEpochDagBytesForHeightFromChain,
    ComputeEpochSeedForHeightFromChain, Block_CalculateAutolykos2Hash,
    Block_RebuildAutolykos2Dag, Autolykos2_LightHash (dead).
  - DAG_BASE_GROWTH and the five DAG_MAX_*_SWING_* / DAG_SWING_PERCENT_DEN
    macros, all now unreachable.

Also: Block_CalculateAutolykos2Hash truncated the height to uint32 while the
light path takes uint64, so the two would have diverged above block 2^32; the
full width is now passed. Added a `dagvote <grow|hold|down>` REPL command and a
progress line during DAG generation, which is tens of seconds at production
sizes.

static_asserts now enforce DAG_MIN_SIZE <= DAG_BASE_SIZE <= DAG_MAX_SIZE and
32-byte alignment, so a misconfigured band fails the build instead of being
silently clamped.

NOTE: DAG_MIN_SIZE (2 GiB), DAG_BASE_SIZE (2 GiB), DAG_MAX_SIZE (8 GiB) and
DAG_EPOCH_STEP (1 GiB) are economic judgements, not derivations, and since the
vote cannot accelerate growth they are the entire upward story. Sanity-check
them before launch.

== Verification ==

  * Penalty: exact table match at d = 4/8/10/50/100/1000, zero within grace,
    saturation at MAX_DEPTH, and identical wall-clock protection when rebuilt
    at TARGET_BLOCK_TIME = 60.
  * DAG recurrence (30 assertions against the real objects): default growth,
    legacy all-zero headers read as grow, strict inequality at exactly 1/2 and
    exactly 7/8, one qualifying epoch freezes but does not shrink, two
    consecutive shrink, sustained shrink repeats, both clamps saturate.
  * Epoch seed: constant across an epoch, equal to hash(block[k*EL - 1]),
    genesis seed in epoch 0, and resolvable at exactly height == EPOCH_LENGTH.
  * Fail-closed: unresolvable params and a zero-byte DAG both reject.
  * Heavy/light equivalence across 3 epochs; heavy refuses a DAG built for the
    wrong epoch or size.
  * Two nodes across 3 epoch boundaries: B reached height 30 purely by
    receiving, then mined blocks A accepted; both fullverify Chain OK, zero
    rejections.
  * Restart mid-epoch: B restarted at height 28 derived epoch 3's seed as
    hash(block[23]) -- the boundary block, not the tip -- and kept producing
    blocks A accepted. This fails before the change.
  * Divergent votes: A voting hold and B voting grow computed identical size
    and seed, confirming the tally is chain-derived, not config-derived.
  * ThreadSanitizer across epoch rollovers under load: no new races.
 2026-07-30 19:32:10 +02:00
dcrubro 4d39614cb5 Fix all ThreadSanitizer-reported races in the node lifecycle (8 reports -> 0) 
Found by running two nodes under -fsanitize=thread through connect -> mine -> broadcast -> clean
exit. None are in consensus code; all are connection setup/teardown.

Stop flags were plain or volatile ints written by one thread and read as a loop condition by
another. volatile stops the compiler hoisting the load but provides neither atomicity nor
ordering, which on arm64 is a real visibility gap, not just a sanitizer complaint. Now _Atomic:
 - net_node_t.maintenanceRunning  (Node_Destroy vs Node_MaintenanceThread)
 - tcp_server_t.isRunning         (TcpServer_Stop vs both accept threads) -- was a bare int, so
                                   the accept loop could legally be hoisted and never see the stop
 - udp_node_t.isRunning           (UdpNode_Stop vs the recv and retry threads)

TcpServer_Stop had a use-after-free, not merely a race: it took clientsMutex only long enough to
read the array pointer, then walked the slots unlocked. An exiting client thread clears its own
slot under that mutex and immediately destroys and frees the connection, so Stop could
RequestClose and pthread_join a freed pointer on any shutdown with an active peer.
 - Stop now requests the close and copies each pthread_t under the mutex, then joins from the
   copied handles, so the connection is never dereferenced outside the lock
 - the client thread's TcpConnection_Destroy/free moved inside the same critical section, which
   closes the window entirely

Client threads that disconnected normally were never joined and leaked their thread resources:
Stop only joins clients still present in the array, and a normal exit removes itself first. Stop
now claims each slot as it copies the handle, so the client thread can tell who owns its join --
it detaches itself if it successfully removed its own slot, and stays joinable if Stop already
claimed it. Both decisions happen under clientsMutex so the cases cannot interleave.

Node_Destroy cleared outbound slots with no lock (via TcpClient_Disconnect) while live inbound
client threads read the same field correctly under outboundLock in Node_HasLiveConnectionTo. The
lock cannot just be held across the destroy, because that path joins an io thread whose
on_disconnect callback takes outboundLock itself. Reworked to detach the connections from their
slots under the lock and tear them down outside it -- the pattern Node_ReapDeadOutbound already
uses in this file. This one only surfaced once the other five were fixed.

Verified: TSan clean over the same run; nodes still converge (height 25, identical tip); shallow
fork still adopts, depth-8 fork still defers on the reorg penalty, and the forced-orphan
regression still reaches full height with zero coinbase rejections.
 2026-07-28 23:07:57 +02:00
dcrubro 42b325d57a Fix self-deadlock on chainLock and fix double-delivery of accepted blocks (+ DUPLICATE enum entry) 2026-07-28 17:22:48 +02:00
dcrubro 1ff2890c0f Fix the reorg system: make rollback survivable, adopt by cumulative work, and enforce the reorg penalty everywhere 
(This is a big one, get ready - I told Claude to write the commit message cause I couldn't be bothered)

Root cause: reorg was broken at every layer and the failures compounded. Verified with the node's own
SKALACOIN_FORCE_ORPHAN_REORG debug mode, which stalled permanently at height 2 on 114 consecutive
coinbase-validation failures. A binary built at HEAD behaves identically, so none of this is a regression —
the reorg path had simply never worked.

Rollback (the keystone):
 - Chain_RollbackToHeight always returned false on any node that had ever saved or loaded its chain, and
   only after it had already truncated the chain and destroyed the balance sheet. Chain_RecomputeRuntimeState
   bails on any header-only block, and Chain_SaveToFile nulls transactions on every in-memory block once
   persisted, so the failure was universal in practice. Both callers treated the false as "nothing happened"
 - Added Chain_BorrowBlockTransactions / Chain_ReturnBlockTransactions, which fall back to the on-disk copy
   when the in-memory block has been compacted to headers
 - Supply is now accumulated in the rollback's existing balance-sheet replay pass instead of a second
   Chain_RecomputeRuntimeState pass
 - Gave Chain_RecomputeRuntimeState the same disk fallback: it had been failing on every restart with an
   existing chain, silently leaving currentSupply/currentReward at whatever came out of chain.meta

Fork choice is now cumulative work, not height:
 - Added Chain_ComputeBlockWork / Chain_ComputeWorkRange / Chain_ComputeBranchWork, computing
   2^256 / (target + 1) per block and summing over a range. Derived on demand from headers — no header,
   chain.meta or wire-format change
 - uint256 only had add/sub/cmp, so added uint256_divide (restoring binary long division),
   uint256_from_be_bytes, uint256_bitwise_not and uint256_is_zero
 - Comparison is strictly greater, so tied tips do not cause two nodes to keep swapping
 - Height-based choice was wrong now that difficulty actually varies: a long low-difficulty branch beat a
   short high-difficulty one

Atomic branch replacement:
 - Added Chain_ReplaceBranch: validate linkage, apply the reorg penalty, compare work, snapshot the outgoing
   blocks, roll back, apply. On any failure the original chain, balance sheet, supply, reward and difficulty
   target are restored. The caller keeps ownership of its blocks in every case — the chain applies copies
 - Split Chain_AddBlock and Chain_RollbackToHeight into locked public wrappers over unlocked internals, so a
   whole branch swap happens under one lock acquisition and Chain_OnTipAdvanced runs once per reorg rather
   than once per block
 - Chain_AddBlock now validates header.prevHash against the tip. It never did — that check lived only in
   Chain_IsValid and the network path, which is exactly why a rollback-then-reapply could splice blocks from
   two different forks into a chain that no longer links up
 - Moved the currentSupply/currentReward update into Chain_AddBlock. Each caller used to do it separately, so
   the orphan-attach and maintenance-thread paths never did, and the next block's coinbase was then validated
   against a stale currentReward and rejected forever. This was the height-2 stall

Reorg penalty (Horizen-style delayed block submission):
 - The penalty was only ever reachable from the manual sync command. The P2P broadcast -> orphan pool ->
   branch adoption path, which is the path an attacker actually uses, had none at all and picked the winner
   by raw height. It is now enforced inside Chain_ReplaceBranch, the single choke point every adoption
   passes through
 - Removed its application to the sync fetch window. The height gap to a peer is not a reorg depth;
   penalising it only throttled honest catch-up, and for gaps of 4-50 it collapsed the window to one block
   per pass, defeating MAX_PARALLEL_FETCHES
 - The depth is stamped once when a branch is first observed (orphan_entry_t.observedAtTipHeight) and never
   recomputed. Re-deriving it from a moving tip never converges: depth and elapsed both grow by one per block
   while penalty(depth) grows faster, so a penalized branch could never be adopted at all
 - The initial-sync exemption now comes from Chain_IsInitialBlockDownload, which uses the local median time
   past over MEDIAN_TIME_SPAN blocks. It used to key off the peer's advertised height, so any peer claiming
   localHeight + INITIAL_SYNC_HEIGHT_DIFF could switch reorg handling off for the whole session. A median
   rather than the tip alone means one backdated block cannot fake it either

Orphan pool (largely rewritten):
 - Added a pool mutex. It had no synchronisation whatsoever while being mutated from the 1 Hz maintenance
   thread, every per-peer TCP thread and the REPL thread; a concurrent insert could realloc the array while a
   scan held a raw element pointer. The lock is never held across a call into chain.c
 - Dedup by block hash, a MAX_ORPHAN_BLOCKS cap with oldest-first eviction, and pruning of entries that can
   no longer apply. Nothing was ever reaped before, and orphans are reachable before the chain-derived
   difficulty check, so this is also the memory-exhaustion fix
 - Candidate branches are now assembled by following prevHash from the fork point. Taking the first orphan
   found at each successive height could interleave blocks from two competing forks into one incoherent branch
 - Fixed rollbackHeight = forkHeight - 1. Chain_RollbackToHeight is exclusive, so every non-genesis adoption
   amputated one block too many and then failed Chain_AddBlock's index check
 - Fixed a block_t wrapper leak on every successful attach (free the wrapper, not Block_Destroy — the chain
   owns the transactions after a shallow copy)
 - Permanently invalid orphans are dropped instead of being retried on every maintenance tick forever

Forks below the tip are now discoverable:
 - A block at blockNumber < chainSize was rejected and freed, so the fork point and the lower half of any
   competing branch were always thrown away and a sub-tip fork could never be learned. Now the hash is
   compared: identical means a duplicate and is ignored, different means it goes to the orphan pool
 - The sync loop probes downwards (RequestForkWindow, bounded by REORG_FETCH_DEPTH and MAX_FORK_PROBE_ROUNDS)
   when it makes no progress while the peer is ahead. That is the only trigger that fires for a genuine
   sub-tip fork, because the old divergence check could only see blocks that had already entered our chain.
   FETCH_BLOCK already answers from the peer's own chain, so no protocol change was needed
 - Removed the rollback-to-height-0 path. "Could not find the parent" used to wipe the entire local chain,
   genesis included, and any peer could trigger it with a single unlinked block

Floating point removed from consensus and network math:
 - Chain_ComputeTargetAtHeight (the difficulty retarget) used double ratio arithmetic, and
   FetchScheduler_ComputeReorgPenaltyBlocks used double/pow/ceil. Both are consensus-critical and are now
   integer only; float results are not reproducible across platforms and compilers, and a single last-digit
   difference in a target or a penalty splits the network
 - The penalty constants became integer rationals (REORG_PENALTY_FACTOR_NUM/DEN, integer EXPONENT and
   REF_BLOCK_TIME) with saturating exponentiation and explicit ceiling division. Output is unchanged:
   penalty(4)=10, penalty(8)=39, penalty(10)=60, penalty(50)=1500, penalty(100)=6000
 - Removed the unused float macros DAG_MAX_UP/DOWN_SWING_PERCENTAGE and the now-dead math.h include from
   constants.h. Both were latent: DAG size feeds PoW verification. Replaced with integer numerator/denominator
   constants and used them at both clamp sites (values verified identical)

Other fixes that were blocking fork propagation:
 - madeProgressOverall was set but never reset, so after one productive pass the "no progress -> stop" guard
   could never fire again and the sync loop could spin forever holding the REPL
 - seenBlocks was inserted before/regardless of a successful send, so a block broadcast while no peer was
   connected was never offered again. It is now recorded only after the block actually goes out
 - Broadcasts relayed to outbound connections only, so in a two-node setup the dialled node never pushed
   anything back and the dialer learned of new blocks only via a manual sync. Inbound peers are now relayed to

Verified with two-node harnesses at a shortened adjustment interval:
 - forced-orphan regression: was height 2 with 114 coinbase rejections, now reaches the peer's height with
   zero rejections and both nodes report Chain OK
 - sub-tip fork at depth 3: the node discovers the fork below its own tip, discards its three blocks, adopts
   the heavier five, and both nodes converge on an identical tip hash
 - deep fork at depth 8: the strictly heavier branch is correctly refused with depth=8 penalty=39 elapsed=0
 - uint256 work arithmetic covered by a standalone test (division, big-endian conversion, monotonicity,
   halved target doubles work)

One issue found along the way: the chain in build/chain_data does not pass fullverify. Block 7683 reverts to
INITIAL_DIFFICULTY where it should carry 0x1f06df14, i.e. it contains blocks mined before 1288a64 landed. A
binary built at HEAD rejects it too, so this is stale data rather than a regression — it needs a wipechain
and a re-mine.
 2026-07-28 00:24:45 +02:00
dcrubro 1288a64977 Fixed the target computation to trigger on syncs - old system would cause difficulty mismatches on syncs, reorgs, etc. 2026-07-27 19:21:39 +02:00
dcrubro c16b88fc5a Add timestamp field to transactions - allows multiple transactions with the same Ins/Outs 2026-07-27 14:34:30 +02:00
dcrubro f9c785b316 Add timestamp field to transactions - allows multiple transactions with the same Ins/Outs 2026-07-27 14:30:06 +02:00
dcrubro 0e90f7d5db Fix multi-homed / IPv6 peer identity: nodes adding themselves and reconnect churn 
Root cause: peers were keyed only by (IP, listen port). An IPv6 host holds several addresses at once, so one node appeared as several peers and could not recognise its own addresses.

Protocol:
 - Added a random per-run node identity (localNodeId), advertised as a length-guarded trailing field in HELLO and ACK_HELLO — backwards compatible with peers that omit it
 - Added peerNodeId to tcp_connection_t; peers are now identified by this rather than by an endpoint
 - ACK_HELLO is now sent before any decision to drop the connection, so a rejected dialer learns whose address it reached instead of retrying forever

Self-connection:
 - Identity match → close the connection and record that endpoint permanently as our own
 - Self-endpoint set preemptively seeded from getifaddrs() at startup, so a node knows its own addresses before ever dialing one
 - Node_ConnectPeer refuses self endpoints, which also kills the echo-back chain that could exhaust connection slots

Duplicate connections and churn:
 - Dedup by identity per direction — one inbound and one outbound per physical peer, regardless of how many addresses it has
 - Moved dial history out of the peer table, so striking a peer no longer resets its connect-retry cooldown (this was the loop engine: strike → re-learn via gossip → redial next tick)
 - Node_HasOtherInboundFrom / Node_HasLiveConnectionTo now ignore connections that are already tearing down, and match on identity as well as endpoint

Gossip hygiene:
 - Never hand a peer its own other addresses in a PEERS reply (matched on identity, not just the socket address)
 - Reject unusable endpoints: link-local without scope id, unspecified, multicast, site-local (loopback stays allowed for local testing)
 - Normalise IPv4-mapped IPv6 so one host cannot occupy two entries

Two bugs found along the way:
 - All nodes drew the same identity — random_eight_byte() comes from srand(time(NULL)), so processes started in the same second produced identical values. Added random_secure_eight_byte() (/dev/urandom) for the identity, and mixed the pid into the seed so connection IDs stop colliding too
 - Identity dedup initially left inbound-only nodes mute — broadcasts traverse outbound connections only, so suppressing a dial-back because an inbound existed would have silenced such a node. Corrected to per-direction

Other:
 - peers output now shows each entry's node identity and the node's own endpoints
 - Added _DEFAULT_SOURCE to the build so getifaddrs() stays visible on glibc
 2026-07-27 14:08:57 +02:00
dcrubro 5aa99ecb01 Change functions IsValidIPv4/v6 to use addrinfo instead of custom logic 2026-07-27 12:31:03 +02:00
dcrubro 3e5c051645 Change functions IsValidIPv4/v6 to use addrinfo instead of custom logic 2026-07-27 12:28:34 +02:00
dcrubro c4e28df46f Fix txpooldetail command not working, add missing commands to list, fix connect command for ipv6 2026-07-27 12:22:43 +02:00
dcrubro 88a9caa46c Reap dead outbound connection slots; strike disconnected peers from discovery. Two related connection-lifecycle fixes on top of the SIGPIPE fix. 
Reclaim outbound slots on peer disconnect:
- TcpClient_ThreadProc fired on_disconnect but never cleared the outbound
  slot, closed the fd, joined the io thread, or freed the connection, so a
  dead peer permanently held its outboundClients[] slot. After MAX_CONS (32)
  churned connections the node could make no new outbound connections, and
  leaked fds/threads/memory. (The inbound side already self-reclaimed.)
- Add a reaper (Node_ReapDeadOutbound) on the maintenance thread: under
  outboundLock it detaches dead (disconnect-notified) slots, then joins the
  io thread and destroys/frees each connection outside the lock.
- Guard against use-after-free with a pin count on tcp_connection_t
  (TcpConnection_Pin/Unpin). The only cross-thread consumer holding a raw
  connection pointer across a blocking op is the `sync` command (via
  Node_GetBestOutboundPeer); it now pins the peer and unpins when done, and
  the reaper skips pinned connections. Discovery's snapshots run on the
  reaper's own thread, so they need no pin.
- Node_GetBestOutboundPeer/GetClientList/GetPeerEndpoints skip
  disconnect-notified connections so a dead peer is never handed out.
- Node_Destroy stops+joins the maintenance thread before tearing down
  outbound clients, so the reaper can't race shutdown.

Strike disconnected peers from the discovery peer list:
- Add NodeDiscovery_RemovePeer + Node_HandlePeerDisconnect: on disconnect,
  remove the peer from the known-peer table once no live connection (inbound
  or outbound) to its listen endpoint remains (Node_HasLiveConnectionTo;
  disconnect-notified conns don't count, so both directions dropping at once
  is handled). Wired into Node_Server_OnDisconnect and Node_Client_OnDisconnect.
 2026-07-26 23:08:21 +02:00
dcrubro f9e3f8cbbb SIGPIPE fix - don't terminate on nodes being nodes 2026-07-23 14:23:38 +02:00
dcrubro 4417095ab5 force one inbound and one outbound between two nodes 2026-07-23 14:00:04 +02:00
dcrubro 21fe73fb01 Add NodeDiscovery: multi-hop peer crawl with UDP-ping preference 
- Implement NodeDiscovery engine: known-peer table (DynArr) with a
  per-tick seed/ping/query/connect state machine driven by the node
  maintenance thread; bounded multi-hop crawl (FANOUT peers per node,
  hop-capped) that connects to reachable peers lowest-ping-first
- Add GET_PEERS/PEERS TCP opcodes for peer-list exchange, handled on
  both inbound and outbound connections
- Measure UDP round-trip time and pass it to the on_pong callback
  (previously the send timestamp was only used for retries)
- Advertise each node's listen port in HELLO/ACK_HELLO and store it
  per-connection, so inbound-only peers and non-default ports are
  discoverable (length-guarded parse; wire-compatible with old peers)
- Wire a udp_node_t + node_discovery_t into net_node_t: init/start in
  Node_Create, tick in the maintenance loop, teardown in Node_Destroy
  (stop UDP before destroying discovery to avoid callback races)
- Add Node_ConnListenEndpoint / Node_GetPeerEndpoints helpers to derive
  peers' listen endpoints (outbound: dialed port; inbound: advertised),
  with IPv4-mapped-IPv6 normalization and IP+port dedup
- Match ping pong/timeout callbacks by peer address (the UDP layer owns
  the nonce), fixing discovered peers stuck UNREACHABLE
- Ping the peer's listen port instead of the ephemeral TCP source port
  (fixes the original stub so pongs actually return)
- Add `peers` CLI command to dump the discovery table (endpoint/hop/
  state/ping)
- Add discovery tunables to constants.h (fanout, max hops, target
  connections, timeouts, caps)
 2026-07-22 23:18:57 +02:00
43 changed files with 8108 additions and 1437 deletions
Expand all files Collapse all files
.gitignore
+1
View File
@@ -6,4 +6,5 @@ main
CMakeFiles/ CMakeFiles/
.vscode/ .vscode/
build/ build/
build*/
.DS_Store .DS_Store
CMakeLists.txt
+376 -6
View File
@@ -9,6 +9,330 @@ set(CMAKE_C_STANDARD 23)
set(CMAKE_C_STANDARD_REQUIRED ON) set(CMAKE_C_STANDARD_REQUIRED ON)
set(CMAKE_C_EXTENSIONS OFF) set(CMAKE_C_EXTENSIONS OFF)
# ---------------------------------------------------------
# Build configuration
#
# cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug # -Og -g3, ASan + UBSan
# cmake -S . -B build-strict -DCMAKE_BUILD_TYPE=Strict # Debug plus -Werror
# cmake -S . -B build-analyzer -DCMAKE_BUILD_TYPE=Analyzer # -Og -g3, GCC -fanalyzer
# cmake -S . -B build-release -DCMAKE_BUILD_TYPE=Release # -O3, _FORTIFY_SOURCE
#
# Debug finds memory bugs at run time, Analyzer finds them at compile time,
# Release survives them. Strict is Debug with warnings promoted to errors — the
# configuration CI should gate on, kept separate so a new warning never blocks
# someone mid-debugging. The warning set is identical in all four; only the
# instrumentation and the error policy differ.
#
# Debug and Analyzer are separate configurations rather than one Debug build
# with everything switched on, because the two tools actively interfere: ASan's
# instrumentation inflates the CFG enough that -fanalyzer exhausts its
# exploration budget and silently stops reporting. Measured on
# src/tcpd/tcpserver.c with GCC 16 — -fanalyzer alone reports the fd leak in
# TcpServer_Init, -fanalyzer plus ASan reports nothing at all.
#
# ThreadSanitizer is deliberately absent: it cannot be combined with ASan, so it
# would need a third configuration of its own.
# ---------------------------------------------------------
set(SKALACOIN_CUSTOM_CONFIGS Strict Analyzer)
set(SKALACOIN_BUILD_TYPES Debug Strict Analyzer Release RelWithDebInfo MinSizeRel)
get_property(SKALACOIN_MULTI_CONFIG GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
if(SKALACOIN_MULTI_CONFIG)
foreach(_cfg IN LISTS SKALACOIN_CUSTOM_CONFIGS)
if(NOT "${_cfg}" IN_LIST CMAKE_CONFIGURATION_TYPES)
list(APPEND CMAKE_CONFIGURATION_TYPES ${_cfg})
endif()
endforeach()
elseif(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Debug CACHE STRING "Build type" FORCE)
message(STATUS "No CMAKE_BUILD_TYPE specified; defaulting to Debug")
endif()
set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS ${SKALACOIN_BUILD_TYPES})
# Strict and Analyzer are custom configurations, so CMake has no built-in flags
# for them. Both inherit Debug's, and imported targets (OpenSSL::Crypto,
# CURL::libcurl) need a mapping because they only ship Debug/Release/NOCONFIG.
foreach(_cfg IN LISTS SKALACOIN_CUSTOM_CONFIGS)
string(TOUPPER ${_cfg} _cfg_upper)
foreach(_lang C CXX)
set(CMAKE_${_lang}_FLAGS_${_cfg_upper} "${CMAKE_${_lang}_FLAGS_DEBUG}"
CACHE STRING "Flags used by the ${_lang} compiler during ${_cfg} builds.")
mark_as_advanced(CMAKE_${_lang}_FLAGS_${_cfg_upper})
endforeach()
foreach(_linker EXE SHARED MODULE STATIC)
set(CMAKE_${_linker}_LINKER_FLAGS_${_cfg_upper} "${CMAKE_${_linker}_LINKER_FLAGS_DEBUG}"
CACHE STRING "Flags used by the linker during ${_cfg} builds.")
mark_as_advanced(CMAKE_${_linker}_LINKER_FLAGS_${_cfg_upper})
endforeach()
set(CMAKE_MAP_IMPORTED_CONFIG_${_cfg_upper} Debug "" Release RelWithDebInfo)
endforeach()
# Debug, Strict and Analyzer share the developer flag set (-Og -g3, warnings).
set(SKALACOIN_IS_DEBUGLIKE "$<OR:$<CONFIG:Debug>,$<CONFIG:Strict>,$<CONFIG:Analyzer>>")
# Debug and Strict are the runtime-instrumented pair; Analyzer must stay clean
# of sanitizers or -fanalyzer goes quiet (see above).
set(SKALACOIN_IS_SANITIZED "$<OR:$<CONFIG:Debug>,$<CONFIG:Strict>>")
# The Strict configuration always errors on warnings; this option additionally
# promotes them in Debug and Analyzer. OFF until src/ is clean under the warning
# set below — until then, build Strict when you want the gate.
option(SKALACOIN_WERROR "Debug/Analyzer: treat warnings as errors (always on in Strict)" OFF)
option(SKALACOIN_ENABLE_SANITIZERS "Debug config: AddressSanitizer + UndefinedBehaviorSanitizer" ON)
option(SKALACOIN_ENABLE_ANALYZER "Analyzer config: the compiler's static analyzer (GCC -fanalyzer)" ON)
# How much of the control-flow path -fanalyzer prints per report. 1 lists only
# the state transitions (opened here / first close here / leaks here), which is
# what you want while triaging; raise it when a report needs the branch-by-branch
# path that explains how it got there. GCC silently accepts out-of-range values,
# so validate here instead.
set(ANALYZER_VERBOSITY 1 CACHE STRING "Analyzer config: -fanalyzer path detail, 0 (terse) to 5 (full)")
set_property(CACHE ANALYZER_VERBOSITY PROPERTY STRINGS 0 1 2 3 4 5)
if(NOT ANALYZER_VERBOSITY MATCHES "^[0-5]$")
message(FATAL_ERROR "ANALYZER_VERBOSITY must be an integer from 0 to 5, got '${ANALYZER_VERBOSITY}'")
endif()
option(SKALACOIN_ENABLE_HARDENING "All configs: stack protector, _FORTIFY_SOURCE, RELRO/NOW, CFI" ON)
option(SKALACOIN_ENABLE_LTO "Optimized configs: link-time optimization" OFF)
include(CheckCCompilerFlag)
include(CheckLinkerFlag)
# Most of the interesting memory diagnostics below only exist from a certain
# GCC/Clang version onwards, and several are architecture specific. Probe every
# flag instead of gating on compiler version, so an older or foreign toolchain
# silently gets the subset it understands rather than failing to configure.
#
# A flag the driver merely tolerates is not a flag that does anything (Clang
# accepts -fstack-clash-protection on arm64 and then ignores it), so the probe
# promotes "argument unused" to an error where the compiler supports that.
check_c_compiler_flag(-Werror=unused-command-line-argument SKALACOIN_HAS_WERROR_UNUSED_ARG)
if(SKALACOIN_HAS_WERROR_UNUSED_ARG)
set(SKALACOIN_FLAG_PROBE_STRICT "-Werror=unused-command-line-argument")
else()
set(SKALACOIN_FLAG_PROBE_STRICT "")
endif()
function(skalacoin_append_supported_c_flags out_var)
set(_accepted ${${out_var}})
set(CMAKE_REQUIRED_FLAGS "${SKALACOIN_FLAG_PROBE_STRICT}")
foreach(_flag IN LISTS ARGN)
string(MAKE_C_IDENTIFIER "SKALACOIN_HAS_CFLAG_${_flag}" _cache_var)
check_c_compiler_flag("${_flag}" ${_cache_var})
if(${_cache_var})
list(APPEND _accepted "${_flag}")
endif()
endforeach()
set(${out_var} "${_accepted}" PARENT_SCOPE)
endfunction()
# Instrumentation such as -fsanitize=address has to reach the linker as well,
# and is only usable if the matching runtime library actually exists, so the
# probe hands the flag to both halves of the try-compile.
#
# Flags are also probed cumulatively, in the order given: some are only legal in
# the presence of an earlier one (-fsanitize=pointer-compare is rejected unless
# -fsanitize=address is already on the command line), and probing it alone would
# quietly drop it.
function(skalacoin_append_supported_instrument_flags out_var)
set(_accepted ${${out_var}})
set(_context "")
foreach(_flag IN LISTS ARGN)
string(MAKE_C_IDENTIFIER "SKALACOIN_HAS_INSTRUMENT_${_flag}" _cache_var)
string(JOIN " " CMAKE_REQUIRED_FLAGS ${SKALACOIN_FLAG_PROBE_STRICT} ${_context})
set(CMAKE_REQUIRED_LINK_OPTIONS ${_context} "${_flag}")
check_c_compiler_flag("${_flag}" ${_cache_var})
if(${_cache_var})
list(APPEND _accepted "${_flag}")
list(APPEND _context "${_flag}")
endif()
endforeach()
set(${out_var} "${_accepted}" PARENT_SCOPE)
endfunction()
function(skalacoin_append_supported_link_flags out_var)
set(_accepted ${${out_var}})
foreach(_flag IN LISTS ARGN)
string(MAKE_C_IDENTIFIER "SKALACOIN_HAS_LDFLAG_${_flag}" _cache_var)
check_linker_flag(C "${_flag}" ${_cache_var})
if(${_cache_var})
list(APPEND _accepted "${_flag}")
endif()
endforeach()
set(${out_var} "${_accepted}" PARENT_SCOPE)
endfunction()
# Warnings for our own C code; the per-config flag sets are applied through
# generator expressions so multi-config generators (Xcode, VS) work too.
set(SKALACOIN_C_WARNINGS "")
set(SKALACOIN_C_FLAGS_DEBUGLIKE "")
set(SKALACOIN_C_FLAGS_OPTIMIZED "")
# The two instrumentation sets, each owned by one configuration and never both.
set(SKALACOIN_SANITIZER_FLAGS "")
set(SKALACOIN_ANALYZER_FLAGS "")
# Instrumentation that must be handed to the linker as well, and that also
# covers the vendored code we link in.
set(SKALACOIN_INSTRUMENT_COMPILE "")
set(SKALACOIN_INSTRUMENT_LINK "")
if(MSVC)
set(SKALACOIN_WERROR_FLAG /WX)
list(APPEND SKALACOIN_C_WARNINGS /W4 /permissive- /sdl)
list(APPEND SKALACOIN_C_FLAGS_DEBUGLIKE /Od /RTC1 /GS)
list(APPEND SKALACOIN_C_FLAGS_OPTIMIZED /O2 /GS /guard:cf)
if(SKALACOIN_WERROR)
list(APPEND SKALACOIN_C_FLAGS_DEBUGLIKE ${SKALACOIN_WERROR_FLAG})
endif()
if(SKALACOIN_ENABLE_ANALYZER)
list(APPEND SKALACOIN_ANALYZER_FLAGS /analyze)
endif()
if(SKALACOIN_ENABLE_SANITIZERS)
list(APPEND SKALACOIN_SANITIZER_FLAGS /fsanitize=address)
endif()
else()
set(SKALACOIN_WERROR_FLAG -Werror)
# Portable warning set, memory-safety first.
skalacoin_append_supported_c_flags(SKALACOIN_C_WARNINGS
-Wall
-Wextra
-Wshadow
-Wconversion
-Wsign-conversion
-Wcast-qual
-Wcast-align
-Wstrict-prototypes
-Wmissing-prototypes
-Wold-style-definition
-Wbad-function-cast
-Wwrite-strings
-Wformat=2
-Wnull-dereference
-Wdouble-promotion
-Wfloat-equal # consensus code must stay integer-only
-Wvla
-Wstack-protector
-Wpointer-arith
-Wundef
-Winit-self
-Wmissing-include-dirs
)
if(CMAKE_C_COMPILER_ID MATCHES "GNU")
# GCC-only diagnostics. The -W*=N forms ask for the strictest level:
# more false positives, but they catch out-of-bounds writes, dangling
# pointers and double frees that -Wall/-Wextra walk straight past.
skalacoin_append_supported_c_flags(SKALACOIN_C_WARNINGS
-Warray-bounds=2
-Wstringop-overflow=4
-Wstringop-truncation
-Wformat-overflow=2
-Wformat-truncation=2
-Wuse-after-free=3
-Wdangling-pointer=2
-Wfree-nonheap-object
-Walloc-zero
-Walloca
-Wduplicated-cond
-Wduplicated-branches
-Wlogical-op
-Wjump-misses-init
-Wtrampolines
-Wflex-array-member-not-at-end
)
# -fanalyzer is a whole-path symbolic execution pass (leaks, double
# free, use-after-free, NULL derefs across function boundaries).
if(SKALACOIN_ENABLE_ANALYZER)
skalacoin_append_supported_c_flags(SKALACOIN_ANALYZER_FLAGS
-fanalyzer-verbosity=${ANALYZER_VERBOSITY}
-fanalyzer
)
endif()
elseif(CMAKE_C_COMPILER_ID MATCHES "Clang")
# Clang-only diagnostics. Clang has no in-compiler equivalent of
# -fanalyzer, so the Analyzer config is simply an uninstrumented Debug
# build here — which is exactly the base `scan-build cmake --build
# build-analyzer` wants.
skalacoin_append_supported_c_flags(SKALACOIN_C_WARNINGS
-Warray-bounds-pointer-arithmetic
-Wconditional-uninitialized
-Wshift-sign-overflow
-Wassign-enum
-Wcomma
-Wloop-analysis
-Wthread-safety
-Wover-aligned
)
endif()
# -Og keeps the code steppable while still running the optimizer passes
# that -Wmaybe-uninitialized / -Wstringop-* rely on; at -O0 those warnings
# go quiet. -fno-omit-frame-pointer buys readable sanitizer backtraces.
skalacoin_append_supported_c_flags(SKALACOIN_C_FLAGS_DEBUGLIKE
-Og
-g3
-fno-omit-frame-pointer
)
if(SKALACOIN_WERROR)
list(APPEND SKALACOIN_C_FLAGS_DEBUGLIKE ${SKALACOIN_WERROR_FLAG})
endif()
if(SKALACOIN_ENABLE_HARDENING)
skalacoin_append_supported_c_flags(SKALACOIN_C_FLAGS_DEBUGLIKE
-fstack-protector-strong
)
# _FORTIFY_SOURCE needs an optimized build to see through the buffer
# sizes, and it fights ASan's interceptors, so it is Release-only.
skalacoin_append_supported_c_flags(SKALACOIN_C_FLAGS_OPTIMIZED
-U_FORTIFY_SOURCE
-D_FORTIFY_SOURCE=3
-fstack-protector-strong
-fstack-clash-protection
)
skalacoin_append_supported_instrument_flags(SKALACOIN_INSTRUMENT_COMPILE
-fcf-protection=full # x86_64 CET
-mbranch-protection=standard # aarch64 BTI/PAC
)
skalacoin_append_supported_link_flags(SKALACOIN_INSTRUMENT_LINK
"LINKER:-z,relro"
"LINKER:-z,now"
"LINKER:-z,noexecstack"
)
endif()
if(SKALACOIN_ENABLE_SANITIZERS)
# Everything that composes with ASan. LeakSanitizer is not listed
# because ASan already includes it where it is supported, and
# -fsanitize=leak cannot be combined with -fsanitize=address.
# -fno-sanitize-recover makes UB abort instead of printing and
# continuing, so a bad shift or overflow cannot be ignored in CI.
# pointer-compare/pointer-subtract additionally need
# ASAN_OPTIONS=detect_invalid_pointer_pairs=2 at run time.
skalacoin_append_supported_instrument_flags(SKALACOIN_SANITIZER_FLAGS
-fsanitize=address
-fsanitize-address-use-after-scope
-fsanitize=pointer-compare
-fsanitize=pointer-subtract
-fsanitize=undefined
-fsanitize=bounds-strict
-fno-sanitize-recover=undefined
-fno-omit-frame-pointer
)
endif()
endif()
# Each instrumentation set belongs to exactly one configuration. Keeping them in
# separate configs is the point of the Analyzer build, so never emit both.
if(SKALACOIN_SANITIZER_FLAGS)
list(APPEND SKALACOIN_INSTRUMENT_COMPILE "$<${SKALACOIN_IS_SANITIZED}:${SKALACOIN_SANITIZER_FLAGS}>")
list(APPEND SKALACOIN_INSTRUMENT_LINK "$<${SKALACOIN_IS_SANITIZED}:${SKALACOIN_SANITIZER_FLAGS}>")
endif()
if(SKALACOIN_ENABLE_LTO)
include(CheckIPOSupported)
check_ipo_supported(RESULT SKALACOIN_IPO_SUPPORTED OUTPUT SKALACOIN_IPO_ERROR)
if(NOT SKALACOIN_IPO_SUPPORTED)
message(WARNING "LTO requested but unsupported by this toolchain: ${SKALACOIN_IPO_ERROR}")
endif()
endif()
find_package(Threads REQUIRED) find_package(Threads REQUIRED)
include(FetchContent) include(FetchContent)
@@ -146,6 +470,11 @@ if(SKALACOIN_ENABLE_AUTOLYKOS2_REF)
add_library(autolykos2_ref STATIC ${AUTOLYKOS2_REF_SOURCES}) add_library(autolykos2_ref STATIC ${AUTOLYKOS2_REF_SOURCES})
target_include_directories(autolykos2_ref PRIVATE ${AUTOLYKOS2_REF_BASE}/include) target_include_directories(autolykos2_ref PRIVATE ${AUTOLYKOS2_REF_BASE}/include)
# Vendored code gets the instrumentation but not our warning set: sanitizers
# only see a bug if the translation unit that owns the memory is compiled
# with them, and this library allocates buffers that our code touches.
target_compile_options(autolykos2_ref PRIVATE "${SKALACOIN_INSTRUMENT_COMPILE}")
target_link_options(autolykos2_ref PRIVATE "${SKALACOIN_INSTRUMENT_LINK}")
# Upstream source uses `malloc/free/exit/EXIT_FAILURE` without including # Upstream source uses `malloc/free/exit/EXIT_FAILURE` without including
# stdlib headers in some C++ translation units. AppleClang can compile this, # stdlib headers in some C++ translation units. AppleClang can compile this,
# while Linux Clang fails. Force-include stdlib.h for C++ in this vendored lib. # while Linux Clang fails. Force-include stdlib.h for C++ in this vendored lib.
@@ -186,7 +515,7 @@ set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib) set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib) set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
foreach(OUTPUTCONFIG DEBUG RELEASE RELWITHDEBINFO MINSIZEREL) foreach(OUTPUTCONFIG DEBUG STRICT ANALYZER RELEASE RELWITHDEBINFO MINSIZEREL)
string(TOUPPER ${OUTPUTCONFIG} OUTPUTCONFIG_UPPER) string(TOUPPER ${OUTPUTCONFIG} OUTPUTCONFIG_UPPER)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_${OUTPUTCONFIG_UPPER} ${CMAKE_BINARY_DIR}/bin) set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_${OUTPUTCONFIG_UPPER} ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_${OUTPUTCONFIG_UPPER} ${CMAKE_BINARY_DIR}/lib) set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_${OUTPUTCONFIG_UPPER} ${CMAKE_BINARY_DIR}/lib)
@@ -220,15 +549,56 @@ endif()
target_include_directories(node PRIVATE target_include_directories(node PRIVATE
${PROJECT_SOURCE_DIR}/include ${PROJECT_SOURCE_DIR}/include
) )
target_compile_options(node PRIVATE # khash is vendored third-party code we cannot fix, and it accounts for half the
-Wall # warnings under Strict. SYSTEM turns -I into -isystem, which suppresses
-Wextra # diagnostics from headers found through it. It needs its own search path: via
-Wpedantic # ${PROJECT_SOURCE_DIR}/include the header resolves through the plain -I above
-g # and stays a normal header, so the sources include it as <khash.h>.
target_include_directories(node SYSTEM PRIVATE
${PROJECT_SOURCE_DIR}/include/khash
) )
target_compile_options(node PRIVATE
"${SKALACOIN_C_WARNINGS}"
"${SKALACOIN_INSTRUMENT_COMPILE}"
"$<${SKALACOIN_IS_DEBUGLIKE}:${SKALACOIN_C_FLAGS_DEBUGLIKE}>"
"$<$<NOT:${SKALACOIN_IS_DEBUGLIKE}>:${SKALACOIN_C_FLAGS_OPTIMIZED}>"
# The static analyzer runs on our C sources only, never on the vendored
# C++ below, and never alongside the sanitizers.
"$<$<CONFIG:Analyzer>:${SKALACOIN_ANALYZER_FLAGS}>"
# Strict is Debug with the warning set turned into a build gate.
"$<$<CONFIG:Strict>:${SKALACOIN_WERROR_FLAG}>"
)
target_link_options(node PRIVATE "${SKALACOIN_INSTRUMENT_LINK}")
if(SKALACOIN_ENABLE_LTO AND SKALACOIN_IPO_SUPPORTED)
set_target_properties(node PROPERTIES
INTERPROCEDURAL_OPTIMIZATION_RELEASE ON
INTERPROCEDURAL_OPTIMIZATION_RELWITHDEBINFO ON
INTERPROCEDURAL_OPTIMIZATION_MINSIZEREL ON
)
endif()
target_compile_definitions(node PRIVATE target_compile_definitions(node PRIVATE
CHAIN_DATA_DIR="${CMAKE_BINARY_DIR}/chain_data" CHAIN_DATA_DIR="${CMAKE_BINARY_DIR}/chain_data"
$<$<BOOL:${SKALACOIN_AUTOLYKOS2_REF_AVAILABLE}>:SKALACOIN_AUTOLYKOS2_REF_AVAILABLE> $<$<BOOL:${SKALACOIN_AUTOLYKOS2_REF_AVAILABLE}>:SKALACOIN_AUTOLYKOS2_REF_AVAILABLE>
$<$<BOOL:1>:_POSIX_C_SOURCE=200809L> $<$<BOOL:1>:_POSIX_C_SOURCE=200809L>
# getifaddrs() (used to learn our own addresses) is a BSD extension, not POSIX; glibc hides it
# unless the default set is requested as well.
$<$<BOOL:1>:_DEFAULT_SOURCE>
) )
set_target_properties(node PROPERTIES OUTPUT_NAME "skalacoin_node") set_target_properties(node PROPERTIES OUTPUT_NAME "skalacoin_node")
# ---------------------------------------------------------
# Configuration summary
# ---------------------------------------------------------
message(STATUS "skalacoin: build type ${CMAKE_BUILD_TYPE}")
message(STATUS "skalacoin: compiler ${CMAKE_C_COMPILER_ID} ${CMAKE_C_COMPILER_VERSION}")
message(STATUS "skalacoin: warnings-as-err ${SKALACOIN_WERROR} (always on in Strict)")
message(STATUS "skalacoin: sanitizers [Debug/Strict] ${SKALACOIN_SANITIZER_FLAGS}")
message(STATUS "skalacoin: static analyzer [Analyzer] ${SKALACOIN_ANALYZER_FLAGS}")
message(STATUS "skalacoin: hardening ${SKALACOIN_ENABLE_HARDENING}")
message(STATUS "skalacoin: LTO ${SKALACOIN_ENABLE_LTO}")
if(CMAKE_BUILD_TYPE STREQUAL "Analyzer" AND NOT SKALACOIN_ANALYZER_FLAGS)
message(STATUS "skalacoin: NOTE - Analyzer config has no static analyzer on "
"${CMAKE_C_COMPILER_ID}; use scan-build over this build tree")
endif()
TODO.txt
+3 -3
View File
@@ -14,9 +14,6 @@ Check if Block FullVerify is actually verifying fully (not missing any condition
A loophole in the reorg penalty system could potentially exist where someone broadcasts blocks one-at-a-time. Determine a solution to this. A loophole in the reorg penalty system could potentially exist where someone broadcasts blocks one-at-a-time. Determine a solution to this.
IPv6 support for the P2P node. Come on guys, it's 2026. RFC 2460 was in 1998. It's about time.
Like if someone is behind NAT, fine, workable. CGNAT? Lmao good luck.
TO TEST: TO TEST:
Implement Horizen's "Reorg Penalty" system to make it harder for the young chain to be attacked by a powerful miner. Implement Horizen's "Reorg Penalty" system to make it harder for the young chain to be attacked by a powerful miner.
@@ -31,3 +28,6 @@ a constant inflation rate of 1.5% per year. It's lower than fiat (USD is ~2.8% p
Move to a GPU algo. RandomX is a good candidate, but CPU mining is not that attractive to anyone but people who actually want to support the project. Move to a GPU algo. RandomX is a good candidate, but CPU mining is not that attractive to anyone but people who actually want to support the project.
Sadly, CPUs won't incentivize people who want to profit, which let's be fair, is the majority of miners. Sadly, CPUs won't incentivize people who want to profit, which let's be fair, is the majority of miners.
IPv6 support for the P2P node. Come on guys, it's 2026. RFC 2460 was in 1998. It's about time.
Like if someone is behind NAT, fine, workable. CGNAT? Lmao good luck.
include/autolykos2/autolykos2.h
+4 -2
View File
@@ -12,7 +12,7 @@ extern "C" {
typedef struct Autolykos2Context Autolykos2Context; typedef struct Autolykos2Context Autolykos2Context;
extern uint64_t autolykos2_sleepBetweenHashOperationsMicroseconds; extern uint64_t g_autolykos2_sleepBetweenHashOperationsMicroseconds;
Autolykos2Context* Autolykos2_Create(void); Autolykos2Context* Autolykos2_Create(void);
void Autolykos2_Destroy(Autolykos2Context* ctx); void Autolykos2_Destroy(Autolykos2Context* ctx);
@@ -32,7 +32,9 @@ bool Autolykos2_Hash(
uint8_t outHash[32] uint8_t outHash[32]
); );
bool Autolykos2_LightHash(const uint8_t* seed, blockchain_t* chain, uint64_t nonce, uint8_t* out); // Derives the DAG lanes it needs straight from the epoch seed, so it needs no DAG allocation and
// stays correct for any height regardless of which epoch a DAG happens to be built for. Produces
// exactly the same hash as Autolykos2_Hash against a DAG generated from the same seed and size.
bool Autolykos2_LightHashAtHeight( bool Autolykos2_LightHashAtHeight(
const uint8_t seed32[32], const uint8_t seed32[32],
const uint8_t* message, const uint8_t* message,
include/balance_sheet.h
+27 -4
View File
@@ -2,11 +2,10 @@
#define BALANCE_SHEET_H #define BALANCE_SHEET_H
#include <stdint.h> #include <stdint.h>
#include <dynarr.h>
#include <stdbool.h> #include <stdbool.h>
#include <stdlib.h> #include <stdlib.h>
#include <stdio.h> #include <stdio.h>
#include <khash/khash.h> #include <khash.h>
#include <crypto/crypto.h> #include <crypto/crypto.h>
#include <block/transaction.h> #include <block/transaction.h>
#include <string.h> #include <string.h>
@@ -16,15 +15,39 @@
typedef struct { typedef struct {
uint8_t address[32]; // For now just the SHA-256 of the public key; allows representation in different encodings (base58, bech32, etc) without changing the underlying data structure uint8_t address[32]; // For now just the SHA-256 of the public key; allows representation in different encodings (base58, bech32, etc) without changing the underlying data structure
uint256_t balance; uint256_t balance;
/**
* Timestamp (unix ms) of the most recent transaction this address SENT that is in the chain.
*
* Replay protection. Without it any historical transaction could be rebroadcast and mined a
* second time, debiting the sender again -- with UTXOs the spent inputs make that impossible,
* but an account model has nothing to stop it. A non-coinbase transaction is only valid if its
* timestamp is strictly greater than this, so a byte-identical replay (same timestamp, same
* hash) can never be included twice. Enforced in Chain_AddBlockLocked; see the note there.
*
* Rebuilt for free by the rollback's balance-sheet replay, so a reorg cannot leave it stale.
* Persisted with the rest of the entry -- note the file has no height marker, so a balance
* sheet that is out of sync with the chain silently resets this to 0 for every account.
**/
uint64_t lastTxTimestamp;
// TODO: Additional things // TODO: Additional things
} balance_sheet_entry_t; } balance_sheet_entry_t;
// KHASH_INIT expands to khash's own implementation, which is not -Wconversion
// clean. -isystem silences the header itself but not code expanded from its
// macros, because the diagnostic is attributed to this line.
#if defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wconversion"
#endif
KHASH_INIT(balance_sheet_map_m, key32_t, balance_sheet_entry_t, 1, hash_key32, eq_key32) KHASH_INIT(balance_sheet_map_m, key32_t, balance_sheet_entry_t, 1, hash_key32, eq_key32)
extern khash_t(balance_sheet_map_m)* sheetMap; #if defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic pop
#endif
extern khash_t(balance_sheet_map_m)* g_sheetMap;
void BalanceSheet_Init(); void BalanceSheet_Init();
int BalanceSheet_Insert(balance_sheet_entry_t entry); int BalanceSheet_Insert(balance_sheet_entry_t entry);
bool BalanceSheet_Lookup(uint8_t* address, balance_sheet_entry_t* out); bool BalanceSheet_Lookup(const uint8_t* address, balance_sheet_entry_t* out);
bool BalanceSheet_SaveToFile(const char* outPath); bool BalanceSheet_SaveToFile(const char* outPath);
bool BalanceSheet_LoadFromFile(const char* inPath); bool BalanceSheet_LoadFromFile(const char* inPath);
void BalanceSheet_Print(); void BalanceSheet_Print();
include/block/block.h
+62 -7
View File
@@ -3,7 +3,7 @@
#include <stdint.h> #include <stdint.h>
#include <openssl/sha.h> #include <openssl/sha.h>
#include <dynarr.h> #include <dlibc/vector.h>
#include <block/transaction.h> #include <block/transaction.h>
#include <stdbool.h> #include <stdbool.h>
#include <string.h> #include <string.h>
@@ -18,26 +18,81 @@ typedef struct {
uint8_t merkleRoot[32]; uint8_t merkleRoot[32];
uint32_t difficultyTarget; // Encoding: [1 byte exponent][3 byte coefficient]; Target = coefficient * 256^(exponent-3) uint32_t difficultyTarget; // Encoding: [1 byte exponent][3 byte coefficient]; Target = coefficient * 256^(exponent-3)
uint8_t version; uint8_t version;
uint8_t reserved[3]; // 3 bytes (Explicit padding for 8-byte alignment) // reserved[0] carries the miner's DAG-size vote (DAG_VOTE_* in constants.h); reserved[1..2] must
// be zero. All three are inside the hashed header, so a vote is committed to by both the
// canonical hash and the PoW hash and cannot be altered after the block is mined.
uint8_t reserved[3];
} block_header_t; } block_header_t;
#pragma pack(pop) #pragma pack(pop)
typedef struct { typedef struct {
block_header_t header; block_header_t header;
DynArr* transactions; // Array of signed_transaction_t, NOTE: Potentially move to a hashmap at some point for quick lookups. vector_t* transactions; // Vector of signed_transaction_t, NOTE: Potentially move to a hashmap at some point for quick lookups.
} block_t; } block_t;
// PoW validity is chain-relative: it needs the epoch DAG size and seed. chain.h includes this
// header, so the tag declared there is forward-declared here to break the cycle.
typedef struct blockchain blockchain_t;
block_t* Block_Create(); block_t* Block_Create();
void Block_CalculateHash(const block_t* block, uint8_t* outHash); void Block_CalculateHash(const block_t* block, uint8_t* outHash);
void Block_CalculateMerkleRoot(const block_t* block, uint8_t* outHash); void Block_CalculateMerkleRoot(const block_t* block, uint8_t* outHash);
void Block_CalculateAutolykos2Hash(const block_t* block, uint8_t* outHash);
bool Block_RebuildAutolykos2Dag(size_t dagBytes, const uint8_t seed32[32]);
void Block_AddTransaction(block_t* block, signed_transaction_t* tx); void Block_AddTransaction(block_t* block, signed_transaction_t* tx);
void Block_RemoveTransaction(block_t* block, uint8_t* txHash); void Block_RemoveTransaction(block_t* block, uint8_t* txHash);
bool Block_HasValidProofOfWork(const block_t* block);
/**
* Autolykos2 PoW hashing.
*
* The heavy variant reads its lanes from the process-global DAG and is a MINING accelerator only;
* the light variant derives the same lanes from the epoch seed on demand. They are bit-for-bit
* equivalent by construction -- Autolykos2_DagGenerate fills lane i with exactly what
* ReadDagLaneFromSeed recomputes for lane i -- so a block mined through either verifies through
* either. Validation always uses the light path: it needs no allocation, which is what keeps the
* DAG a miner requirement rather than a full-node memory requirement, and it stays correct for
* blocks from earlier epochs (the heavy path can only ever answer for whichever epoch the global
* DAG was last built for).
**/
bool Block_EnsureAutolykos2Dag(uint64_t epochIndex, size_t dagBytes, const uint8_t seed32[32]);
// Fails rather than answering from a DAG built for a different epoch, size OR SEED, so it can
// never silently hash against the wrong lanes. The seed matters because a reorg changes it while
// leaving the epoch index and size unchanged.
bool Block_PowHashHeavy(const block_t* block, uint64_t epochIndex, size_t dagBytes,
const uint8_t seed32[32], uint8_t outHash[32]);
bool Block_PowHashLight(const block_t* block, size_t dagBytes, const uint8_t seed32[32], uint8_t outHash[32]);
// PoW check against explicitly supplied epoch parameters, for callers that resolve them once and
// then iterate (the miner). Returns false if the hash cannot be computed -- never treat an
// uncomputable proof as valid.
bool Block_HasValidProofOfWorkWithParams(const block_t* block, uint64_t epochIndex,
size_t dagBytes, const uint8_t seed32[32]);
// PoW check that resolves the epoch parameters for the block's own height from `chain`.
bool Block_HasValidProofOfWork(const block_t* block, blockchain_t* chain);
// Header vote field is a recognised value and the unused reserved bytes are zero.
bool Block_HasValidVote(const block_t* block);
bool Block_AllTransactionsValid(const block_t* block); bool Block_AllTransactionsValid(const block_t* block);
bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinbaseAmount, uint64_t* outTotalFees); bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinbaseAmount, uint64_t* outTotalFees);
bool Block_IsFullyValid(const block_t* block);
/**
* Self-contained validity: merkle root, transactions, vote encoding, non-empty. Needs no chain, so
* it is meaningful for ANY block, including one on a branch we do not have.
*
* This is what the receive path checks. Proof of work is deliberately NOT checked there, because
* PoW is only meaningful relative to the branch a block belongs to: the epoch seed is the last
* block of the previous epoch on ITS OWN branch. Validating a competing branch's block against our
* epoch seed does not merely fail to resolve -- when the two chains diverge before the boundary it
* resolves to the WRONG seed and rejects a perfectly valid block, which made any fork spanning an
* epoch boundary impossible to assemble.
*
* Chain_AddBlock verifies proof of work at the moment a block joins the chain, where the branch
* context is real. That, not the receive path, is what enforces the invariant.
**/
bool Block_HasValidStructure(const block_t* block);
// Full check including chain-relative PoW. Only meaningful for a block that extends `chain`.
bool Block_IsFullyValid(const block_t* block, blockchain_t* chain);
void Block_ShutdownPowContext(void); void Block_ShutdownPowContext(void);
void Block_Destroy(block_t* block); void Block_Destroy(block_t* block);
void Block_Print(const block_t* block); void Block_Print(const block_t* block);
include/block/chain.h
+122 -4
View File
@@ -2,18 +2,48 @@
#define CHAIN_H #define CHAIN_H
#include <block/block.h> #include <block/block.h>
#include <dynarr.h> #include <dlibc/vector.h>
#include <stdlib.h> #include <stdlib.h>
#include <stdio.h> #include <stdio.h>
#include <stdbool.h> #include <stdbool.h>
#include <string.h> #include <string.h>
#include <pthread.h>
#include <uint256.h> #include <uint256.h>
#include <storage/block_table.h> #include <storage/block_table.h>
#include <balance_sheet.h> #include <balance_sheet.h>
// One entry of the memoised DAG size recurrence, one per epoch. See Chain_DagParamsForHeight.
typedef struct { typedef struct {
DynArr* blocks; uint64_t sizeBytes; // DAG size used by every block whose height falls in this epoch
bool downQualified; // this epoch's own votes met the down supermajority
} dag_epoch_state_t;
// Tagged so block.h can forward-declare it: PoW validity depends on the chain (it needs the epoch
// seed), but chain.h includes block.h, so the tag is what breaks the cycle.
typedef struct blockchain {
// Vector of block_t, stored by value. Each element owns its `transactions` vector, which the
// element destructor frees whenever a block leaves the chain (rollback, clear, destroy).
vector_t* blocks;
size_t size; size_t size;
/**
* Memoised DAG size recurrence: a pure cache of a function of the block headers, extended
* lazily and dropped whenever anything at or below the tip changes (every epoch's size depends
* on the votes of every epoch before it). It lives on the chain rather than in a global because
* a second, header-only blockchain_t is built to re-verify historical PoW, and the two must not
* share a cache.
*
* `dagEpochsComputed` counts valid `sizeBytes` entries. `downQualified` is only filled in for
* an epoch once the *following* entry has been computed, so it is valid on
* [0, dagEpochsComputed - 1).
*
* Guarded by `dagCacheLock`, which is always taken AFTER `g_chainLock` and is never held across a
* call back into chain.c.
**/
dag_epoch_state_t* dagEpochs;
size_t dagEpochsComputed;
size_t dagEpochsCapacity;
pthread_mutex_t dagCacheLock;
} blockchain_t; } blockchain_t;
blockchain_t* Chain_Create(); blockchain_t* Chain_Create();
@@ -28,7 +58,55 @@ void Chain_Wipe(blockchain_t* chain);
// Returns true on success. // Returns true on success.
bool Chain_RollbackToHeight(blockchain_t* chain, size_t height); bool Chain_RollbackToHeight(blockchain_t* chain, size_t height);
// Recompute `currentSupply` and `currentReward` from the in-memory chain blocks. /**
* Atomically replace the blocks at [forkHeight, tip] with `newBlocks` (ascending, `count` of them).
*
* The swap happens only if the candidate branch is properly linked, has strictly more cumulative
* work, and has served its Horizen delayed-submission penalty. `observedAtTipHeight` is the local
* tip height at which the branch was FIRST seen and must not be recomputed as the chain grows --
* see the comment in the implementation. The initial-block-download exemption is decided inside,
* from local state only, so no caller can switch the penalty off.
*
* `bypassPenalty` skips the delay check ONLY. It exists for an explicit operator action (`sync
* force`) on a node whose chain is known to be the wrong one -- the penalty is served by local
* chain growth, so a node that is neither mining nor stale enough to count as catching up cannot
* clear it on its own. It must never be reachable from anything a peer says; work comparison,
* linkage and atomicity are still enforced, so this cannot adopt a branch that is not heavier.
*
* On any failure the original chain, balance sheet, supply and reward are restored and false is
* returned. The caller keeps ownership of `newBlocks` in every case: the chain applies copies.
**/
bool Chain_ReplaceBranch(blockchain_t* chain,
size_t forkHeight,
block_t** newBlocks,
size_t count,
uint64_t observedAtTipHeight,
bool bypassPenalty);
// True when this node is catching up rather than following the tip (empty chain, or a median
// block time far in the past). Used to exempt initial sync from the reorg penalty.
bool Chain_IsInitialBlockDownload(blockchain_t* chain);
// Penalty in blocks of local chain growth before a branch forking `reorgDepth` blocks back may be
// adopted. Thin wrapper over FetchScheduler_ComputeReorgPenaltyBlocks, for callers that only
// want to report it.
uint64_t Chain_ReorgPenaltyForDepth(uint64_t reorgDepth);
/**
* Replay guard: true if every non-coinbase transaction in `block` is newer than its own sender's
* last included transaction, and newer than that same sender's earlier transactions in this block.
*
* Reads the balance sheet's per-account `lastTxTimestamp` (see balance_sheet.h). Senders are
* considered independently -- one account's transactions say nothing about another's ordering, so
* an ordinary block full of different senders always passes. Coinbase is exempt.
*
* Exposed rather than inlined so this can be tested directly; Chain_AddBlockLocked calls it as part
* of block validation, which is what makes it apply to mining, sync, broadcast, orphan attach and
* reorg alike.
**/
bool Chain_BlockRespectsSenderOrdering(const block_t* block);
// Recompute `g_currentSupply` and `g_currentReward` from the in-memory chain blocks.
// Returns true on success and updates runtime state globals. // Returns true on success and updates runtime state globals.
bool Chain_RecomputeRuntimeState(blockchain_t* chain); bool Chain_RecomputeRuntimeState(blockchain_t* chain);
@@ -41,6 +119,46 @@ bool Chain_LoadFromFile(blockchain_t* chain, const char* dirpath, uint256_t* out
bool Chain_LoadBlockFromFile(const char* dirpath, uint64_t blockNumber, bool loadTransactions, block_t** outBlock, size_t* outTxCount); bool Chain_LoadBlockFromFile(const char* dirpath, uint64_t blockNumber, bool loadTransactions, block_t** outBlock, size_t* outTxCount);
// Difficulty // Difficulty
uint32_t Chain_ComputeNextTarget(blockchain_t* chain, uint32_t currentTarget); // Retarget for the block at `height`, measured over the window [height - INTERVAL, height - 1].
// `chain` must hold blocks 0..height-1. Takes no locks; safe to call while holding `g_chainLock`.
uint32_t Chain_ComputeTargetAtHeight(blockchain_t* chain, uint64_t height, uint32_t currentTarget);
// The consensus-required difficultyTarget for the block at `height`, derived from the chain alone.
// Takes no locks; safe to call while holding `g_chainLock`.
uint32_t Chain_GetTargetForHeight(blockchain_t* chain, uint64_t height);
// Refresh runtime state derived from the chain tip (difficulty target, epoch DAG).
// Call after any change to the tip. Must NOT be called while holding `g_chainLock`.
void Chain_OnTipAdvanced(blockchain_t* chain);
// DAG
/**
* The Autolykos2 DAG size and epoch seed that the block at `blockHeight` must be hashed against.
*
* This is the single source of truth for both, so the mining path and the verification path cannot
* drift apart. Size follows the default-grow recurrence gated by the miner votes in
* `header.reserved[0]` (see the DAG band in constants.h); the seed is epoch-aligned -- epoch 0 uses
* the genesis seed, epoch k uses the hash of the last block of epoch k-1 -- so it is constant for
* the whole epoch rather than changing every block.
*
* Requires the chain to hold every block below the start of `blockHeight`'s epoch, which is always
* true when validating or mining a block at that height. Returns false if it cannot produce both
* values; callers MUST treat that as an invalid proof rather than falling back to a default.
*
* Takes `g_chainLock` for reading internally. Must NOT be called while holding it.
**/
bool Chain_DagParamsForHeight(blockchain_t* chain, uint64_t blockHeight,
size_t* outDagBytes, uint8_t outSeed[32]);
// Work
// Expected number of hashes to satisfy `difficultyTargetBits`, i.e. 2^256 / (target + 1).
bool Chain_ComputeBlockWork(uint32_t difficultyTargetBits, uint256_t* outWork);
// Summed work of the chain's blocks over the half-open range [from, to).
// Takes no locks; safe to call while holding `g_chainLock`.
bool Chain_ComputeWorkRange(blockchain_t* chain, size_t from, size_t to, uint256_t* outWork);
// Summed work of a candidate branch that is not (yet) part of the chain.
bool Chain_ComputeBranchWork(block_t** blocks, size_t count, uint256_t* outWork);
#endif #endif
include/block/transaction.h
+6 -1
View File
@@ -20,9 +20,14 @@ static inline bool Address_IsCoinbase(const uint8_t address[32]) {
return true; return true;
} }
// 160 bytes total for v1 // 168 bytes total for v1
#pragma pack(push, 1) // Ensure no padding for consistent file storage #pragma pack(push, 1) // Ensure no padding for consistent file storage
typedef struct { typedef struct {
uint64_t timestamp; // Unix timestamp in MILLISECONDS (get_current_time_ms). Two of a sender's
// transactions must have strictly increasing timestamps -- see
// lastTxTimestamp in balance_sheet.h. Millisecond resolution is what makes
// an exact collision mean 'byte-identical replay' rather than 'two real
// transactions that happened to coincide'.
uint64_t fee; // Rewarded to the miner; can be zero, but the miner may choose to ignore transactions with very low fees uint64_t fee; // Rewarded to the miner; can be zero, but the miner may choose to ignore transactions with very low fees
uint64_t amount1; uint64_t amount1;
uint64_t amount2; uint64_t amount2;
include/constants.h
+176 -104
View File
@@ -13,6 +13,18 @@
#define MAX_CONS 32 // Some baseline for now #define MAX_CONS 32 // Some baseline for now
#define LISTEN_PORT 9393 #define LISTEN_PORT 9393
#define ECHO_PEERS 1 // If non-zero, automatically attempt to connect back to any inbound peers (helps form bidirectional peering) #define ECHO_PEERS 1 // If non-zero, automatically attempt to connect back to any inbound peers (helps form bidirectional peering)
// Node discovery
#define DISCOVERY_FANOUT 2 // "A couple" - how many peers to query per round, and how many new peers to accept per PEERS response (keeps the crawl spread out)
#define DISCOVERY_MAX_HOPS 3 // How many hops away from us we keep crawling
#define DISCOVERY_TARGET_CONNECTIONS 8 // Desired outbound connection count discovery tries to reach (bounded by MAX_CONS)
#define DISCOVERY_MAX_KNOWN_PEERS 256 // Cap on the known-peer table size
#define DISCOVERY_PEERS_RESPONSE_CAP 8 // Max endpoints we put in a single PEERS response
#define DISCOVERY_MAX_PINGS_PER_TICK 8 // Cap on UDP pings sent per discovery tick
#define DISCOVERY_PING_TIMEOUT_MS 5000ULL // Backstop: a PINGED peer with no pong for this long is marked unreachable
#define DISCOVERY_PING_REFRESH_MS 60000ULL // Re-ping a reachable peer after this long to refresh its latency
#define DISCOVERY_QUERY_INTERVAL_MS 15000ULL // Minimum interval between GET_PEERS to the same peer
#define DISCOVERY_CONNECT_RETRY_MS 30000ULL // Minimum interval between connect attempts to the same discovered peer
#define TCP_THREAD_STACK_SIZE (512 * 1024) // 512 KB. We could get away with like 128 KB since it's mostly just recv bufs, but it's good having some breathing room. #define TCP_THREAD_STACK_SIZE (512 * 1024) // 512 KB. We could get away with like 128 KB since it's mostly just recv bufs, but it's good having some breathing room.
// This is also for client threads. The server has the default (~8 MB on POSIX). // This is also for client threads. The server has the default (~8 MB on POSIX).
@@ -21,8 +33,26 @@
#define DIFFICULTY_ADJUSTMENT_INTERVAL 3840 // Every 3840 blocks (roughly every 4 days with a 90 second block time) #define DIFFICULTY_ADJUSTMENT_INTERVAL 3840 // Every 3840 blocks (roughly every 4 days with a 90 second block time)
// Max adjustment per is x2. So if blocks are coming in too fast, the difficulty will at most double every 24 hours, and vice versa if they're coming in too slow. // Max adjustment per is x2. So if blocks are coming in too fast, the difficulty will at most double every 24 hours, and vice versa if they're coming in too slow.
#define TARGET_BLOCK_TIME 90 // Target block time in seconds #define TARGET_BLOCK_TIME 90 // Target block time in seconds
//#define INITIAL_DIFFICULTY 0x1f0c1422 // Default compact target used by Autolykos2 PoW (This is ridiculously low) // The retarget measures the span between the FIRST and LAST block of the window, which is one fewer
#define INITIAL_DIFFICULTY 0x1f1b7c51 // This takes 90s on my machine with a single thread, good for testing // interval than the window has blocks, and divides by it. Two blocks is the minimum that leaves a
// non-zero span. See Chain_ComputeTargetAtHeight.
static_assert(DIFFICULTY_ADJUSTMENT_INTERVAL >= 2,
"DIFFICULTY_ADJUSTMENT_INTERVAL must span at least one block interval");
#define INITIAL_DIFFICULTY 0x1f0c1422 // Default compact target used by Autolykos2 PoW (This is ridiculously low)
//#define INITIAL_DIFFICULTY 0x1f1b7c51 // Ridiculously low difficulty for testing.
// Mining
// The timestamp lives in the header the PoW hashes, so the miner restamps it while searching rather
// than keeping the one stamped when the search started. Two things fall out of that: a block carries
// the time it was actually found instead of a timestamp that is a whole block time stale on average,
// and every restamp is a fresh search space, so the nonce sweep starts over from 0 and never has to
// walk out to keep finding untried candidates. It costs nothing to throw the old nonce range away --
// each attempt is independent, so the work already done was never getting any closer.
static const uint64_t MINING_TIMESTAMP_REFRESH_MS = 2ULL; // Don't restamp for a drift smaller than this
// Reading the clock once per hash would be wasted work next to a memory-hard hash, so the check is
// batched. Note this, not the refresh interval, is what actually bounds accuracy once a batch of
// hashes takes longer than MINING_TIMESTAMP_REFRESH_MS -- keep it small enough that it doesn't.
static const uint64_t MINING_TIMESTAMP_CHECK_NONCES = 16ULL;
// Sync / Reorg tuning constants // Sync / Reorg tuning constants
// Timeouts and retry/backoff behavior for block fetches during sync (milliseconds) // Timeouts and retry/backoff behavior for block fetches during sync (milliseconds)
@@ -31,14 +61,79 @@ static const int MAX_SYNC_RETRIES = 4; // retry attempts per block fetch
static const uint64_t SYNC_BACKOFF_BASE_MS = 200ULL; // base backoff in ms (exponential) static const uint64_t SYNC_BACKOFF_BASE_MS = 200ULL; // base backoff in ms (exponential)
// Parallelism // Parallelism
static const int MAX_PARALLEL_FETCHES = 8; // concurrent block fetches during windowed sync static const int MAX_PARALLEL_FETCHES = 8; // concurrent block fetches during windowed sync
// Heuristic: if peer is this many blocks ahead, treat as initial sync // How far below a detected divergence we ask a peer for blocks, so the orphan pool has enough of
static const uint64_t INITIAL_SYNC_HEIGHT_DIFF = 50ULL; // the competing branch to locate the fork point by prevHash linkage.
static const uint64_t REORG_FETCH_DEPTH = 128ULL;
// How many times one `sync` will probe downwards for a fork point before giving up, so a peer on a
// permanently incompatible chain cannot keep us looping.
static const int MAX_FORK_PROBE_ROUNDS = 3;
// Reorg penalty configuration (used to penalize peers reporting higher heights but with delayed work) // Reorg penalty configuration (Horizen-style delayed block submission penalty).
// A branch forking B blocks below our tip is held for penalty(B) blocks of local chain growth
// before it may be adopted, so a rented-hashrate attacker has to sustain the attack publicly
// instead of winning by dumping a privately mined branch.
//
// penalty(B) = ceil(FACTOR_NUM/FACTOR_DEN * B^EXPONENT * REF_BLOCK_TIME / TARGET_BLOCK_TIME)
//
// The block-time ratio is REF/TARGET, not TARGET/REF. penalty() counts BLOCKS, so the wall-clock
// protection is penalty(B) * TARGET_BLOCK_TIME ~= B^EXPONENT * REF_BLOCK_TIME: TARGET_BLOCK_TIME
// cancels and the protection is block-time-independent. See fetch_scheduler.c.
//
// Expressed as integer rationals on purpose: this feeds fork choice, so it must evaluate
// identically on every node. Floating point is not acceptable here.
static const uint64_t REORG_PENALTY_GRACE_BLOCKS = 3ULL; // allow small reorgs without penalty static const uint64_t REORG_PENALTY_GRACE_BLOCKS = 3ULL; // allow small reorgs without penalty
static const double REORG_PENALTY_FACTOR = 1.0; // base scaling factor (theta) static const uint64_t REORG_PENALTY_FACTOR_NUM = 1ULL; // base scaling factor (theta), numerator
static const double REORG_PENALTY_EXPONENT = 2.0; // exponent p in penalty ~ B^p static const uint64_t REORG_PENALTY_FACTOR_DEN = 1ULL; // base scaling factor (theta), denominator
static const double REORG_PENALTY_REF_BLOCK_TIME = 150.0; // reference block time in seconds used by original scheme static const uint32_t REORG_PENALTY_EXPONENT = 2U; // exponent p in penalty ~ B^p
static const uint64_t REORG_PENALTY_REF_BLOCK_TIME = 150ULL; // reference block time in seconds used by original scheme
// Beyond this depth the penalty saturates. At the configured parameters penalty(1000) is already
// ~1.67M blocks (~4.75 years at a 90s block time), so this only exists to keep the arithmetic away
// from overflow rather than to bound the penalty in any meaningful sense.
static const uint64_t REORG_PENALTY_MAX_DEPTH = 1000ULL;
/**
* Mempool transaction timestamp policy. LOCAL POLICY, NOT CONSENSUS.
*
* These govern what this node is willing to hold and relay; a block containing a transaction that
* violates either is still accepted. That separation is deliberate -- a node with a skewed clock
* must not be able to fork itself off the network over an admission rule.
*
* A too-OLD timestamp needs no rule here: the per-account replay guard (see balance_sheet.h) already
* refuses anything at or below a sender's last included transaction.
**/
// Refuse to admit a transaction dated further ahead than this of OUR OWN CLOCK. Measured against
// the clock and not against the chain tip on purpose: on a quiet chain the tip can be hours old, and
// judging "future" against it would refuse honest transactions exactly when blocks are sparse.
static const uint64_t TX_MAX_FUTURE_DRIFT_MS = 2ULL * 60ULL * 60ULL * 1000ULL; // 2 hours
// Drop transactions older than this from the mempool, so it is not inflated by junk that will never
// be mined. Roughly the ~4 days DIFFICULTY_ADJUSTMENT_INTERVAL spans, but expressed in milliseconds
// so it does not drift if the block time changes.
static const uint64_t TX_EXPIRY_MS = 4ULL * 24ULL * 60ULL * 60ULL * 1000ULL; // 4 days
// Upper bound on pooled orphan blocks. Orphans are accepted before the chain-derived difficulty
// check (that lives in Chain_AddBlock, which orphans only reach on attach), so without a cap a
// peer can push blocks at an arbitrary height until the node runs out of memory.
static const size_t MAX_ORPHAN_BLOCKS = 512U;
// A node whose chain tip is older than this many target block times is catching up rather than
// following the tip, and is exempt from the reorg penalty (Horizen does the same via
// IsInitialBlockDownload). Determined purely from local state, so an unverified peer cannot
// trigger the exemption by claiming a large height.
//
// This is also the ONLY way a non-mining node rejoins the network after ending up on a minority
// fork: the penalty is served by local chain growth, and a node that does not mine has no way to
// grow except by adopting the very branch the penalty is gating. It therefore has to be short
// enough that such a node recovers in minutes rather than half a day.
//
// 20 block times is ~30 minutes at a 90s target, far beyond normal Poisson block spacing (a gap
// that long has probability ~e^-20), so a node that is genuinely following the tip will not trip
// it. Note the exemption is all-or-nothing -- once in IBD a node accepts a reorg of any depth --
// so lowering this further widens that hole; it is the number to revisit if deep reorgs ever get
// used against an idle node.
static const uint64_t IBD_TIP_AGE_BLOCKS = 20ULL;
// Number of trailing blocks whose median timestamp is used for the age test above. Using a median
// rather than the tip alone means a single miner cannot backdate one block to fake being in IBD.
#define MEDIAN_TIME_SPAN 11U
// Reward schedule acceleration: 1 means normal-speed progression. // Reward schedule acceleration: 1 means normal-speed progression.
#define EMISSION_ACCELERATION_FACTOR 1ULL #define EMISSION_ACCELERATION_FACTOR 1ULL
@@ -53,40 +148,78 @@ static const double REORG_PENALTY_REF_BLOCK_TIME = 150.0; // reference block tim
// Keep this at 20 to match the canonical curve shape against a 2^64 atomic supply cap. // Keep this at 20 to match the canonical curve shape against a 2^64 atomic supply cap.
#define MONERO_EMISSION_SPEED_FACTOR 20U #define MONERO_EMISSION_SPEED_FACTOR 20U
// Future Autolykos2 constants: // Autolykos2 epoch / DAG constants.
#define EPOCH_LENGTH 350000 // ~1 year at 90s #define EPOCH_LENGTH 350000 // ~1 year at 90s
#define DAG_BASE_GROWTH (1ULL << 30) // 1 GB per epoch, adjusted by acceleration #define DAG_GENESIS_SEED 0x00 // Epoch 0's seed is all zeroes; epoch k's seed is the hash of the last
//#define DAG_BASE_SIZE (6ULL << 30) // 6 GB, adjusted per cycle based off DAG_BASE_GROWTH // block of epoch k-1, so it is unpredictable until that block is mined.
#define DAG_BASE_SIZE (1ULL << 30) // TEMPORARY FOR TESTING
// Swings - calculated as MIN(percentage, absolute GB) to prevent absurd swings from low hashrate or very large DAG growth
#define DAG_MAX_UP_SWING_PERCENTAGE 1.15 // 15%
#define DAG_MAX_DOWN_SWING_PERCENTAGE 0.90 // 10%
#define DAG_MAX_UP_SWING_GB (2ULL << 30) // 2 GB
#define DAG_MAX_DOWN_SWING_GB (1ULL << 30) // 1 GB
#define DAG_GENESIS_SEED 0x00 // Genesis seed is zeroes, every epoch's seed is the hash of the previous block, therefore unpredictable until the block is mined
/** /**
* Each epoch has 2 phases, connected logarithmically: * DAG size band and the miner signal that moves within it.
* - Phase 1: Aggressive DAG growth (target is ~75% of the max cap) to kick out any ASICs, 30k blocks (roughly 1 month) *
* - Phase 2: Stable DAG growth (target is the max cap) to provide a stable environment for GPU miners, 320k blocks (roughly 11 months) * Growth is the DEFAULT: the size walks up by DAG_EPOCH_STEP every epoch unless miners actively
* brake it. There is deliberately no "grow faster" vote -- every signal a miner can express only
* slows the walk or reverses it. That is what makes the scheme safe against pool capture: under
* stratum-style pooled mining the pool builds the header, so it controls its share of the vote, and
* a pool that wanted a larger DAG to price smaller miners out simply has no lever to pull. The
* entire upward trajectory is set by DAG_EPOCH_STEP and DAG_MAX_SIZE, i.e. by release, not by vote.
*
* DAG_MIN_SIZE is the ASIC-resistance floor: it must stay above the on-die SRAM an ASIC could
* economically carry, because *this constant*, not the vote, is what secures the property. No vote
* outcome can go below it. DAG_MAX_SIZE is the intended destination rather than an emergency bound,
* since the DAG reaches it on its own -- pick it as the largest DAG miners should ever hold.
*
* NOTE: these three sizes are economic judgements, not derivations. Sanity-check them before
* launch. DAG_BASE_SIZE was previously commented as an intended 6 GiB; it now has to sit inside
* the band (see the static_assert below). Lowering the DAG for a test run means lowering
* DAG_MIN_SIZE too, not just DAG_BASE_SIZE.
**/ **/
#define DAG_MIN_SIZE (2ULL << 30) // 2 GiB -- ASIC-resistance floor
#define DAG_BASE_SIZE (2ULL << 30) // epoch 0 size
#define DAG_MAX_SIZE (8ULL << 30) // 8 GiB -- intended destination, ~6 unbraked years from base
#define DAG_EPOCH_STEP (1ULL << 30) // 1 GiB drift per epoch, in either direction
// Vote thresholds as integer numerator/denominator pairs, never float literals: this feeds PoW
// verification, so every node must reach the same verdict. The tests cross-multiply rather than
// divide, so there is no rounding to disagree on.
#define DAG_BRAKE_NUM 1ULL
#define DAG_BRAKE_DEN 2ULL // brake growth when hold+down votes exceed 1/2 of the epoch
#define DAG_DOWN_NUM 7ULL
#define DAG_DOWN_DEN 8ULL // shrink when down votes exceed 7/8 of the epoch, two epochs running
// reserved[0] of the block header carries the vote. 0 must mean GROW: the point of this shape is
// that inaction produces growth, so a miner that knows nothing about the vote contributes to the
// intended default instead of silently freezing the schedule.
#define DAG_VOTE_GROW 0u // default -- let the schedule run
#define DAG_VOTE_HOLD 1u // brake: stop growing
#define DAG_VOTE_DOWN 2u // reverse: shrink (needs a sustained supermajority to take effect)
#define DAG_VOTE_MAX DAG_VOTE_DOWN
static_assert(DAG_MIN_SIZE <= DAG_BASE_SIZE && DAG_BASE_SIZE <= DAG_MAX_SIZE,
"DAG_BASE_SIZE must start inside [DAG_MIN_SIZE, DAG_MAX_SIZE]");
static_assert(DAG_MIN_SIZE % 32ULL == 0ULL && DAG_MAX_SIZE % 32ULL == 0ULL &&
DAG_BASE_SIZE % 32ULL == 0ULL && DAG_EPOCH_STEP % 32ULL == 0ULL,
"Autolykos2 lane addressing requires every DAG size to be a multiple of 32");
static_assert(DAG_EPOCH_STEP > 0ULL, "DAG_EPOCH_STEP must be positive or the DAG can never move");
static const uint64_t M_CAP = 18446744073709551615ULL; // Max uint64 static const uint64_t M_CAP = 18446744073709551615ULL; // Max uint64
static const uint64_t TAIL_EMISSION = 750000000000ULL; // 0.75 coins per block floor static const uint64_t TAIL_EMISSION = 750000000000ULL; // 0.75 coins per block floor
// No max supply. Instead of halving, it'll follow a more gradual, Monero-like emission curve. // No max supply. Instead of halving, it'll follow a more gradual, Monero-like emission curve.
// Phase 3: update once per effective epoch and keep a fixed per-block reward for that epoch. // Phase 3: update once per effective epoch and keep a fixed per-block reward for that epoch.
static inline uint64_t GetInflationRateReward(uint256_t currentSupply, blockchain_t* chain) { //
if (!chain || !chain->blocks) { return 0x00; } // Invalid // The *AtHeight variants take the height directly and never call Chain_Size/Chain_GetBlockCopy, so
size_t height = Chain_Size(chain); // they are safe to call from inside a g_chainLock critical section. g_chainLock is a non-recursive
// pthread_rwlock_t: taking it for reading while this thread already holds it for writing deadlocks
// as soon as another thread is queued for the write lock.
static inline uint64_t GetInflationRateRewardAtHeight(uint256_t currentSupply, uint64_t height) {
const uint64_t effectiveEpochLength = const uint64_t effectiveEpochLength =
(EPOCH_LENGTH / EMISSION_ACCELERATION_FACTOR) > 0 (EPOCH_LENGTH / EMISSION_ACCELERATION_FACTOR) > 0
? (EPOCH_LENGTH / EMISSION_ACCELERATION_FACTOR) ? (EPOCH_LENGTH / EMISSION_ACCELERATION_FACTOR)
: 1; : 1;
if (height == 0) { if (height == 0) {
currentReward = TAIL_EMISSION; g_currentReward = TAIL_EMISSION;
return currentReward; return g_currentReward;
} }
if (height % effectiveEpochLength == 0) { if (height % effectiveEpochLength == 0) {
@@ -109,25 +242,27 @@ static inline uint64_t GetInflationRateReward(uint256_t currentSupply, blockchai
} }
uint64_t inflationPerBlock = quotient.limbs[0]; uint64_t inflationPerBlock = quotient.limbs[0];
currentReward = (inflationPerBlock > TAIL_EMISSION) ? inflationPerBlock : TAIL_EMISSION; g_currentReward = (inflationPerBlock > TAIL_EMISSION) ? inflationPerBlock : TAIL_EMISSION;
return currentReward; return g_currentReward;
} }
return (currentReward > TAIL_EMISSION) ? currentReward : TAIL_EMISSION; return (g_currentReward > TAIL_EMISSION) ? g_currentReward : TAIL_EMISSION;
} }
static inline uint64_t CalculateBlockReward(uint256_t currentSupply, blockchain_t* chain) { static inline uint64_t GetInflationRateReward(uint256_t currentSupply, blockchain_t* chain) {
if (!chain || !chain->blocks) { return 0x00; } // Invalid if (!chain || !chain->blocks) { return 0x00; } // Invalid
return GetInflationRateRewardAtHeight(currentSupply, (uint64_t)Chain_Size(chain));
}
static inline uint64_t CalculateBlockRewardAtHeight(uint256_t currentSupply, uint64_t height) {
const uint64_t effectivePhase1Blocks = const uint64_t effectivePhase1Blocks =
(PHASE1_TARGET_BLOCKS / EMISSION_ACCELERATION_FACTOR) > 0 (PHASE1_TARGET_BLOCKS / EMISSION_ACCELERATION_FACTOR) > 0
? (PHASE1_TARGET_BLOCKS / EMISSION_ACCELERATION_FACTOR) ? (PHASE1_TARGET_BLOCKS / EMISSION_ACCELERATION_FACTOR)
: 1; : 1;
const uint64_t height = (uint64_t)Chain_Size(chain);
// After the phase-one target horizon, only floor/inflation schedule applies. // After the phase-one target horizon, only floor/inflation schedule applies.
if (height >= effectivePhase1Blocks) { if (height >= effectivePhase1Blocks) {
return GetInflationRateReward(currentSupply, chain); return GetInflationRateRewardAtHeight(currentSupply, height);
} }
if (currentSupply.limbs[1] > 0 || if (currentSupply.limbs[1] > 0 ||
@@ -136,7 +271,7 @@ static inline uint64_t CalculateBlockReward(uint256_t currentSupply, blockchain_
currentSupply.limbs[0] >= M_CAP) currentSupply.limbs[0] >= M_CAP)
{ {
// Post-Monero phase with unlimited supply: floor/inflation schedule only. // Post-Monero phase with unlimited supply: floor/inflation schedule only.
return GetInflationRateReward(currentSupply, chain); return GetInflationRateRewardAtHeight(currentSupply, height);
} }
const uint64_t generated = currentSupply.limbs[0]; const uint64_t generated = currentSupply.limbs[0];
@@ -168,80 +303,17 @@ static inline uint64_t CalculateBlockReward(uint256_t currentSupply, blockchain_
} }
// Phase 2 + 3: floor and epoch inflation updates. // Phase 2 + 3: floor and epoch inflation updates.
return GetInflationRateReward(currentSupply, chain); return GetInflationRateRewardAtHeight(currentSupply, height);
} }
// Hashing DAG static inline uint64_t CalculateBlockReward(uint256_t currentSupply, blockchain_t* chain) {
#include <math.h> if (!chain || !chain->blocks) { return 0x00; } // Invalid
static inline size_t CalculateTargetDAGSize(blockchain_t* chain) { return CalculateBlockRewardAtHeight(currentSupply, (uint64_t)Chain_Size(chain));
// Base size plus (base growth * difficulty factor), adjusted by acceleration
if (!chain || !chain->blocks) { return 0; } // Invalid
uint64_t height = (uint64_t)Chain_Size(chain);
if (height < EPOCH_LENGTH) {
return DAG_BASE_SIZE;
}
// Get the height - EPOCH_LENGTH block and the last block;
block_t* lastBlock = NULL;
block_t* epochStartBlock = NULL;
if (!Chain_GetBlockCopy(chain, Chain_Size(chain) - 1, &lastBlock) || !lastBlock) {
if (lastBlock) Block_Destroy(lastBlock);
return 0;
}
if (!Chain_GetBlockCopy(chain, (size_t)(Chain_Size(chain) - 1 - EPOCH_LENGTH), &epochStartBlock) || !epochStartBlock) {
Block_Destroy(lastBlock);
if (epochStartBlock) Block_Destroy(epochStartBlock);
return 0;
}
int64_t difficultyDelta = (int64_t)epochStartBlock->header.difficultyTarget - (int64_t)lastBlock->header.difficultyTarget;
int64_t growth = (DAG_BASE_GROWTH * difficultyDelta); // Can be negative if difficulty has decreased, which is why we use int64_t
// Clamp
if (growth > 0) {
// Difficulty increased -> Clamp the UPWARD swing
int64_t maxUp = (int64_t)((DAG_BASE_SIZE * 15) / 100); // 15%
if (growth > maxUp) growth = maxUp;
if (growth > (int64_t)DAG_MAX_UP_SWING_GB) growth = DAG_MAX_UP_SWING_GB;
} else {
// Difficulty decreased -> Clamp the DOWNWARD swing
int64_t maxDown = (int64_t)((DAG_BASE_SIZE * 10) / 100); // 10%
if (-growth > maxDown) growth = -maxDown;
if (-growth > (int64_t)DAG_MAX_DOWN_SWING_GB) growth = -(int64_t)DAG_MAX_DOWN_SWING_GB;
}
int64_t targetSize = (int64_t)DAG_BASE_SIZE + growth;
if (targetSize <= 0) {
Block_Destroy(lastBlock);
Block_Destroy(epochStartBlock);
return 0;
}
size_t out = (size_t)targetSize;
Block_Destroy(lastBlock);
Block_Destroy(epochStartBlock);
return out;
} }
static inline void GetNextDAGSeed(blockchain_t* chain, uint8_t outSeed[32]) { // Hashing DAG: see Chain_DagParamsForHeight in block/chain.h. Both the size and the epoch seed are
if (!chain || !chain->blocks || !outSeed) { return; } // Invalid // derived from the chain by that one function, so the mining and verification paths cannot drift
uint64_t height = (uint64_t)Chain_Size(chain); // apart. The previous CalculateTargetDAGSize/GetNextDAGSeed pair lived here, took g_chainLock
// internally, was not epoch-aligned, and disagreed with the verifier's own copy in main.c.
if (height < EPOCH_LENGTH) {
memset(outSeed, DAG_GENESIS_SEED, 32);
return;
}
block_t* prevBlock = NULL;
if (!Chain_GetBlockCopy(chain, Chain_Size(chain) - 1, &prevBlock) || !prevBlock) {
memset(outSeed, 0x00, 32); // Fallback to zeroes if we can't get the previous block for some reason; The caller should treat this as an error if height >= EPOCH_LENGTH
if (prevBlock) Block_Destroy(prevBlock);
return;
}
Block_CalculateHash(prevBlock, outSeed);
Block_Destroy(prevBlock);
}
#endif #endif
include/dlibc/set.h
+1141
View File
File diff suppressed because it is too large Load Diff
include/dlibc/vector.h
+887
View File
@@ -0,0 +1,887 @@
/*
vector.h - Vectors for C
Version: 1.0.0
Description:
Header-only library for vectors in C. Memory safe against the mistakes it can
detect, but not thread-safe (you must implement your own synchronization
mechanisms).
Designed for C23 onwards, but should be compatible with older standards (tested
from C99 onwards).
Guarantees:
- A vector_t must never be duplicated by copying the struct. Two vector_t values
sharing one data pointer both believe they own it, and whichever is destroyed
or moved from first leaves the other holding freed memory. Copying the pointer
(vector_t* b = a;) is fine - that is one vector with two names, and it must be
destroyed exactly once.
- To hand a vector's data to another vector, use vector_move(). The source is
emptied, freed and its pointer set to NULL, so there is only ever one owner.
- To get a second, independent vector with the same contents, use
vector_deep_copy(), which copies the data as well as the vector itself. Deep
copies are only available for vectors without an element destructor, as a
byte-wise copy of owning elements would result in a double free.
- No gaps in the raw data array.
- When a vector is resized, the data is reallocated to a new memory location,
so any pointers into the old data are invalidated. There is no way to
preserve them; re-fetch with vector_get() after the resize.
- Order of elements is preserved.
- It is safe to pass a pointer into a vector's own data as the source element
of vector_push_back(), vector_insert() and vector_set(). Each of them handles
the aliasing in whichever way suits it, so the value survives the reallocation
or the shifting of elements. On a vector with an element destructor this is
rejected with -1 instead, as a byte-wise copy would leave two slots owning the
same memory. To duplicate an owning element, copy what it owns yourself and
push that.
- vector_take_at() and vector_take_back() transfer ownership of the element to the
caller, and do not call the destructor on it. It is the caller's responsibility
to free whatever the element owns.
Element destructors:
A vector may be given an element destructor with vector_set_destructor(). It is
called for every element that leaves the vector, i.e. by vector_pop_back(),
vector_pop_at(), vector_set() (on the element being overwritten), vector_clear()
and vector_destroy().
The destructor is NOT called by vector_take_back() or vector_take_at(), as these
functions transfer ownership of the element to the caller. It is the caller's
responsibility to free whatever the element owns.
The destructor receives a pointer to the element's slot inside the vector's data
array - NOT a pointer that was returned by malloc(). It must free whatever the
element owns, and must never free the pointer it was handed, as that memory
belongs to the vector. For a vector of char* this means:
void free_str(void* element) { free(*(char**)element); }
vector_t* vec = vector_create(sizeof(char*));
vector_set_destructor(vec, free_str);
// ... use the vector ...
vector_destroy(&vec); // Takes the address of your pointer, and NULLs it
A warning regarding misuse:
The vector manages its own buffer correctly: no leaks, no double frees of the data
array, no use of stale pointers internally. It cannot reason about what your
elements own — that is what the destructor is for, and it is your responsibility
to set one and to avoid duplicating owned pointers between slots.
License:
MIT License
Copyright (c) 2026 DcruBro
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#ifndef VECTOR_H
#define VECTOR_H
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
typedef void (*vector_destructor_t)(void* element); // Function pointer type for element destructor
/*
A vector. Do not construct, copy or modify this struct directly - the fields are
visible only because the functions below are inline. Create vectors with
vector_create() and use the accessors. A vector_t that did not come from
vector_create() is not a valid vector, and the functions will reject it where
they can and misbehave where they cannot.
*/
typedef struct {
size_t size; // Number of elements in the vector
size_t capacity; // Allocated capacity of the vector
size_t element_size; // Size of each element in the vector
void *data; // Pointer to the raw data array
vector_destructor_t destructor; // Function pointer to the element destructor
} vector_t;
/* Forward declarations */
static inline int vector_is_aliased(const vector_t* vec, const void* ptr);
static inline vector_t* vector_create(size_t element_size);
static inline int vector_set_destructor(vector_t* vec, vector_destructor_t destructor);
static inline vector_destructor_t vector_get_destructor(const vector_t* vec);
static inline int vector_reserve(vector_t* vec, size_t new_capacity);
static inline int vector_grow(vector_t* vec);
static inline int vector_prune(vector_t* vec);
static inline int vector_push_back(vector_t* vec, const void* element);
static inline int vector_pop_back(vector_t* vec);
static inline int vector_pop_at(vector_t* vec, size_t index);
static inline int vector_take_at(vector_t* vec, size_t index, void* out);
static inline int vector_take_back(vector_t* vec, void* out);
static inline int vector_insert(vector_t* vec, size_t index, const void* element);
static inline int vector_clear(vector_t* vec);
static inline int vector_set(vector_t* vec, size_t index, const void* element);
static inline int vector_is_empty(const vector_t* vec);
static inline void* vector_front(vector_t* vec);
static inline const void* vector_front_const(const vector_t* vec);
static inline void* vector_back(vector_t* vec);
static inline const void* vector_back_const(const vector_t* vec);
static inline void* vector_get(vector_t* vec, size_t index);
static inline const void* vector_get_const(const vector_t* vec, size_t index);
static inline size_t vector_size(const vector_t* vec);
static inline size_t vector_capacity(const vector_t* vec);
static inline size_t vector_element_size(const vector_t* vec);
static inline const void* vector_as_c_array(const vector_t* vec);
static inline void* vector_as_c_array_mutable(vector_t* vec);
static inline int vector_move(vector_t* dest, vector_t** src);
static inline vector_t* vector_deep_copy(const vector_t* vec);
static inline int vector_destroy(vector_t** vec);
/*
@brief Checks whether a pointer points inside the vector's own data array. Used internally to make the write functions safe against self-referential input.
@param vec A pointer to the vector to check against.
@param ptr The pointer to check.
@return 1 if the pointer lies within the vector's allocated buffer, 0 otherwise.
@attention This is an internal helper. You are not expected to call it directly, but it is harmless if you do.
*/
static inline int vector_is_aliased(const vector_t* vec, const void* ptr) {
if (!vec || !ptr || !vec->data) {
return 0; // Nothing to alias
}
// Compared as integers rather than pointers, as comparing pointers into
// different objects is not well defined.
uintptr_t base = (uintptr_t)vec->data;
uintptr_t end = base + (vec->capacity * vec->element_size);
uintptr_t target = (uintptr_t)ptr;
return target >= base && target < end;
}
/*
@brief Creates a new vector with the specified element size.
@param element_size The size of each element in the vector. Call with sizeof(type)
@return A pointer to the newly created vector, or NULL if allocation fails. The vector lives on the heap.
@attention The vector must be destroyed with vector_destroy() to free its memory. Failing to do so will result in a memory leak.
@attention The vector's initial capacity is set to 10. If you want to change the initial capacity globally, set the DLIBC_VECTOR_INITIAL_CAPACITY macro before including this header file. The initial capacity must be greater than 0.
@attention The vector's element size must be greater than 0. If you pass 0, the function will return NULL.
@attention The vector is created without an element destructor. If your elements own memory of their own, set one with vector_set_destructor().
*/
static inline vector_t* vector_create(size_t element_size) {
if (element_size == 0) {
return NULL; // Invalid element size
}
#ifndef DLIBC_VECTOR_INITIAL_CAPACITY
size_t initial_capacity = 10; // Default initial capacity
#else
#if DLIBC_VECTOR_INITIAL_CAPACITY <= 0
#error "DLIBC_VECTOR_INITIAL_CAPACITY must be greater than 0"
#endif
size_t initial_capacity = DLIBC_VECTOR_INITIAL_CAPACITY; // Use the macro if defined
#endif
if (SIZE_MAX / element_size < initial_capacity) {
return NULL; // Prevent overflow
}
vector_t* vec = (vector_t*)malloc(sizeof(vector_t));
if (!vec) {
return NULL; // Allocation failed
}
vec->size = 0;
vec->capacity = initial_capacity;
vec->element_size = element_size;
vec->destructor = NULL; // Initialize destructor to NULL
vec->data = malloc(vec->capacity * vec->element_size);
if (!vec->data) {
free(vec);
return NULL; // Allocation failed
}
return vec;
}
/*
@brief Sets the destructor function for the vector's elements. This function will be called on each element as it leaves the vector, allowing for custom cleanup of dynamically allocated memory within the elements.
@param vec A pointer to the vector for which to set the destructor.
@param destructor A function pointer to the destructor function, or NULL to remove the current one. The function should take a single void* parameter, which will be a pointer to the element to be destroyed.
@return 0 on success, -1 if the vector is NULL.
@attention If you do not set a destructor function, the vector will not automatically free any dynamically allocated memory within its elements when it is destroyed. You must ensure that you free any such memory manually before destroying the vector to avoid memory leaks.
@attention The destructor is passed a pointer to the element's slot inside the vector's data array, not a pointer returned by malloc(). It must free what the element owns and must never free the pointer it is given. For a vector of char*, the destructor body is free(*(char**)element);
@attention This assumes that your destructor function is valid and safe to call. If it isn't, bad things will happen and it will not be nice to watch.
@attention Set the destructor before adding any elements. Setting it on a vector that already holds elements is allowed, but elements that were removed beforehand will not have been destroyed.
*/
static inline int vector_set_destructor(vector_t* vec, vector_destructor_t destructor) {
if (!vec) {
return -1; // Invalid vector
}
vec->destructor = destructor;
return 0; // Success
}
/*
@brief Gets the destructor function currently set on the vector.
@param vec A pointer to the vector whose destructor is to be retrieved.
@return The vector's destructor function pointer, or NULL if the vector is NULL or has no destructor set.
*/
static inline vector_destructor_t vector_get_destructor(const vector_t* vec) {
if (!vec) {
return NULL; // Invalid vector
}
return vec->destructor;
}
/*
@brief Ensures the vector has room for at least the specified number of elements.
@param vec A pointer to the vector for which to reserve space.
@param new_capacity The minimum capacity the vector should have.
@return 0 on success, -1 if the vector is NULL, its element size is zero, or allocation fails.
@attention Capacity never decreases. If the vector already has room, this is a no-op and reports success.
@attention If the buffer does grow, it is reallocated, so any pointers into the vector's data are invalidated. Use vector_prune() to release unused memory.
*/
#if defined(__GNUC__) && !defined(__clang__) && __GNUC__ >= 10
// GCC's -fanalyzer loses track of new_data once it is stored through a vector that the caller
// reached via a struct member (holder->vec), and reports the grown buffer as leaked when this
// returns. The buffer is owned by vec->data and released by vector_destroy(), so the report is a
// false positive. Scoped to this function only, so genuine leaks elsewhere are still reported.
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wanalyzer-malloc-leak"
#endif
static inline int vector_reserve(vector_t* vec, size_t new_capacity) {
if (!vec) {
return -1; // Invalid vector
}
if (vec->element_size == 0) {
return -1; // Not a usable vector; also guards the division below
}
if (new_capacity <= vec->capacity) {
return 0; // Already have the room. Never shrink
}
if (new_capacity > SIZE_MAX / vec->element_size) {
return -1; // Prevent overflow
}
void* new_data = realloc(vec->data, new_capacity * vec->element_size);
if (!new_data) {
return -1; // Allocation failed
}
vec->data = new_data;
vec->capacity = new_capacity;
return 0; // Success
}
#if defined(__GNUC__) && !defined(__clang__) && __GNUC__ >= 10
#pragma GCC diagnostic pop
#endif
/*
@brief Grows the vector's capacity to make room for at least one more element. Used internally by the functions that add elements.
@param vec A pointer to the vector to grow.
@return 0 on success, -1 if the vector is NULL or allocation fails.
@attention This is an internal helper. You are not expected to call it directly.
*/
static inline int vector_grow(vector_t* vec) {
if (!vec) {
return -1; // Invalid vector
}
if (vec->capacity > SIZE_MAX / 2) {
return -1; // Prevent overflow
}
size_t new_capacity = vec->capacity > 0 ? vec->capacity * 2 : 1; // Double the capacity, or set to 1 if it was 0
return vector_reserve(vec, new_capacity);
}
/*
@brief Prunes the vector to free unused memory. If the vector's size is less than its capacity, this function will reallocate the vector's data array to match its size, freeing any unused memory.
@param vec A pointer to the vector to be pruned.
@return 0 on success, -1 if the vector is NULL or allocation fails.
@attention After calling this function, the vector's capacity will be equal to its size. Any pointers to the old data will be invalidated.
@attention Capacity may never drop below 1, even if the vector is empty.
*/
static inline int vector_prune(vector_t* vec) {
if (!vec) {
return -1; // Invalid vector
}
if (vec->element_size == 0) {
return -1; // Not a usable vector; also guards the division below
}
if (vec->size < vec->capacity) {
size_t new_capacity = vec->size > 0 ? vec->size : 1; // Ensure capacity is at least 1
if (new_capacity > SIZE_MAX / vec->element_size) {
return -1; // Prevent overflow
}
void* new_data = realloc(vec->data, new_capacity * vec->element_size);
if (!new_data) {
return -1; // Allocation failed
}
vec->data = new_data;
vec->capacity = new_capacity;
}
return 0; // Success
}
/*
@brief Adds an element to the end of the vector.
@param vec A pointer to the vector to which the element will be added.
@param element A pointer to the element to be added. The element will be copied into the vector's data array.
@return 0 on success, -1 if the vector is NULL, the element is NULL, if the element is an owning element that is aliased in the vector and a destructor is set, or if reservation fails.
@attention The element must be a pointer to a valid memory location containing data of the same type as the vector's element type. The vector will make a copy of the data, so the original element can be modified or freed after this function returns.
@attention The element may point into the vector's own data array. If growing the vector moves the data, the pointer is followed to its new location automatically.
*/
static inline int vector_push_back(vector_t* vec, const void* element) {
if (!vec || !element) {
return -1; // Invalid vector or element
}
if (vec->destructor && vector_is_aliased(vec, element)) {
return -1; // Refuse to push an owning element that lives inside the vector
}
if (vec->size >= vec->capacity) {
// If the element lives inside our own data array, remember where it sits
// so that we can find it again after the data has been reallocated.
// This must happen before the growth, while the old buffer is still valid
size_t offset = 0;
int aliased = vector_is_aliased(vec, element);
if (aliased) {
// Subtracted as integers, for the same reason vector_is_aliased() compares them that
// way. Pointer subtraction is only defined within one object, which the check above
// guarantees, but a static analyzer cannot see through the integer comparison and
// reports the subtraction as undefined behaviour
offset = (size_t)((uintptr_t)element - (uintptr_t)vec->data);
}
if (vector_grow(vec) != 0) {
return -1; // Failed to grow the vector
}
if (aliased) {
element = (const char*)vec->data + offset;
}
}
// Copy the new element into the vector's data array
memmove((char*)vec->data + (vec->size * vec->element_size), element, vec->element_size);
vec->size++;
return 0; // Success
}
/*
@brief Removes the last element from the vector.
@param vec A pointer to the vector from which the element will be removed.
@return 0 on success, -1 if the vector is NULL or empty.
@attention After calling this function, the vector's size will be reduced by one. The memory occupied by the removed element will not be freed automatically unless a destructor is set (in which case, the destructor will be called on the element being removed if it is set). Use vector_take_back() to retrieve the last element, remove it from the vector, and NOT call the destructor on it (hands ownership to the caller).
*/
static inline int vector_pop_back(vector_t* vec) {
if (!vec || vec->size == 0) {
return -1; // Invalid vector or empty vector
}
if (vec->destructor) {
void* element = (char*)vec->data + ((vec->size - 1) * vec->element_size);
vec->destructor(element); // Call the destructor for the last element
}
vec->size--;
return 0; // Success
}
/*
@brief Removes the element at the specified index from the vector.
@param vec A pointer to the vector from which the element will be removed.
@param index The index of the element to be removed.
@return 0 on success, -1 if the vector is NULL or the index is out of bounds.
@attention After calling this function, the vector's size will be reduced by one. The memory occupied by the removed element will not be freed automatically unless a destructor is set (in which case, the destructor will be called on the element being removed if it is set). Use vector_take_at() to retrieve the element at the specified index, remove it from the vector, and NOT call the destructor on it (hands ownership to the caller).
*/
static inline int vector_pop_at(vector_t* vec, size_t index) {
if (!vec || index >= vec->size) {
return -1; // Invalid vector or index out of bounds
}
if (vec->destructor) {
void* element = (char*)vec->data + (index * vec->element_size);
vec->destructor(element); // Call the destructor for the element to be removed
}
// Move elements after the index one position to the left
if (index + 1 < vec->size) {
memmove((char*)vec->data + (index * vec->element_size),
(char*)vec->data + ((index + 1) * vec->element_size),
(vec->size - index - 1) * vec->element_size);
}
vec->size--;
return 0; // Success
}
/*
@brief Removes the element at the specified index and transfers ownership of it to the caller.
@param vec A pointer to the vector from which the element will be taken.
@param index The index of the element to take.
@param out A pointer to a buffer of at least vector_element_size(vec) bytes, which the element is copied into.
@return 0 on success, -1 if the vector is NULL, out is NULL, out points into the vector's own data, or the index is out of bounds.
@attention The element destructor is deliberately NOT called. Whatever the element owns becomes the caller's responsibility to free.
@attention out must not point into the vector's own data array. Doing so would leave two slots owning the same memory and is rejected with -1.
*/
static inline int vector_take_at(vector_t* vec, size_t index, void* out) {
if (!vec || index >= vec->size || !out) {
return -1; // Invalid vector, index out of bounds, or output pointer is NULL
}
if (vector_is_aliased(vec, out)) {
return -1; // Refuse to take an element into a pointer that lives inside the vector
}
void* slot = (char*)vec->data + (index * vec->element_size);
memcpy(out, slot, vec->element_size); // Copy the element to the output pointer
// Move elements after the index one position to the left
if (index + 1 < vec->size) {
memmove(slot,
(char*)vec->data + ((index + 1) * vec->element_size),
(vec->size - index - 1) * vec->element_size);
}
vec->size--;
return 0; // Success
}
/*
@brief Removes the last element from the vector and transfers ownership of it to the caller.
@param vec A pointer to the vector from which the element will be taken.
@param out A pointer to a buffer of at least vector_element_size(vec) bytes, which the element is copied into.
@return 0 on success, -1 if the vector is NULL, empty, if out aliases the vector's data, or out is NULL.
@attention The element destructor is deliberately NOT called. Whatever the element owns becomes the caller's responsibility to free.
*/
static inline int vector_take_back(vector_t* vec, void* out) {
if (!vec || vec->size == 0 || !out) {
return -1; // Invalid vector, empty vector, or output pointer is NULL
}
return vector_take_at(vec, vec->size - 1, out); // Take the last element
}
/*
@brief Inserts an element at the specified index in the vector.
@param vec A pointer to the vector into which the element will be inserted.
@param index The index at which to insert the element.
@param element A pointer to the element to be inserted. The element will be copied into the vector's data array.
@return 0 on success, -1 if the vector is NULL, the element is NULL, the index is out of bounds, if the element is an owning element that is aliased in the vector and a destructor is set, or if reservation fails.
@attention After calling this function, the vector's size will be increased by one and every element from the index onwards will have moved one position to the right. Any pointers into the vector are invalidated.
@attention The element may point into the vector's own data array. Its value is copied aside first, so that neither the reallocation nor the shifting of elements can corrupt it.
*/
static inline int vector_insert(vector_t* vec, size_t index, const void* element) {
if (!vec || !element || index > vec->size) {
return -1; // Invalid vector, element, or index out of bounds
}
if (vec->destructor && vector_is_aliased(vec, element)) {
return -1; // Refuse to insert an owning element that lives inside the vector
}
// If the element lives inside our own data array, take a copy of it now. Both
// the reservation below and the shifting of elements would otherwise destroy
// the value before it can be read
void* temp = NULL;
if (vector_is_aliased(vec, element)) {
temp = malloc(vec->element_size);
if (!temp) {
return -1; // Allocation failed
}
memcpy(temp, element, vec->element_size);
element = temp;
}
if (vec->size >= vec->capacity) {
if (vector_grow(vec) != 0) {
free(temp);
return -1; // Failed to grow the vector
}
}
// Move elements after the index one position to the right
memmove((char*)vec->data + ((index + 1) * vec->element_size),
(char*)vec->data + (index * vec->element_size),
(vec->size - index) * vec->element_size);
// Copy the new element into the vector's data array at the specified index
memcpy((char*)vec->data + (index * vec->element_size), element, vec->element_size);
vec->size++;
free(temp); // Harmless if no copy was needed, as temp is NULL in that case
return 0; // Success
}
/*
@brief Clears all elements from the vector.
@param vec A pointer to the vector to be cleared.
@return 0 on success, -1 if the vector is NULL.
@attention After calling this function, the vector's size will be zero. The memory occupied by the elements will not be freed automatically unless a destructor is set.
@attention The vector's capacity is left untouched, so the vector may be refilled without reallocating. Call vector_prune() afterwards if you want the memory back.
*/
static inline int vector_clear(vector_t* vec) {
if (!vec) {
return -1; // Invalid vector
}
if (vec->destructor) {
for (size_t i = 0; i < vec->size; ++i) {
void* element = (char*)vec->data + (i * vec->element_size);
vec->destructor(element); // Call the destructor for each element
}
}
vec->size = 0;
return 0; // Success
}
/*
@brief Sets the element at the specified index in the vector, overwriting whatever was there before.
@param vec A pointer to the vector in which to set the element.
@param index The index of the element to set.
@param element A pointer to the element to set. The element will be copied into the vector's data array.
@return 0 on success, -1 if the vector is NULL, the element is NULL, if the element is an owning element that is aliased in the vector and a destructor is set, or if the index is out of bounds.
@attention This never grows the vector. The index must be below the current size.
@attention If a destructor is set, it is called on the element being overwritten before the new value is copied in.
@attention Setting an element to itself is a no-op and reports success.
@attention On a vector without a destructor, the element may point into the
vector's own data array; the copy handles any overlap. On a vector with a
destructor this is refused with -1, as it would leave two slots owning the
same memory.
*/
static inline int vector_set(vector_t* vec, size_t index, const void* element) {
if (!vec || !element || index >= vec->size) {
return -1; // Invalid vector, element, or index out of bounds
}
void* slot = (char*)vec->data + (index * vec->element_size);
if (slot == element) {
return 0; // Setting the element to itself, no action needed
}
if (vec->destructor && vector_is_aliased(vec, element)) {
return -1; // Refuse to set an owning element that lives inside the vector
}
if (vec->destructor) {
vec->destructor(slot); // Call the destructor for the element being overwritten
}
memmove(slot, element, vec->element_size); // memmove, as the element may overlap the slot
return 0; // Success
}
/*
@brief Checks if the vector is empty.
@param vec A pointer to the vector to check.
@return 1 if the vector is empty, 0 if it is not empty.
@attention If the vector is NULL, this function will return 1 (is empty) to indicate that the vector is invalid.
*/
static inline int vector_is_empty(const vector_t* vec) {
if (!vec) {
return 1; // Invalid vector
}
return vec->size == 0;
}
/*
@brief Gets a pointer to the first element in the vector.
@param vec A pointer to the vector from which to get the element.
@return A pointer to the first element in the vector, or NULL if the vector is NULL or empty.
@attention The returned pointer is a generic. You should cast it to the appropriate type before using it. The pointer will become invalid if the vector is resized or destroyed.
*/
static inline void* vector_front(vector_t* vec) {
if (!vec || vec->size == 0) {
return NULL; // Invalid vector or empty vector
}
return vec->data;
}
/*
@brief Gets a const pointer to the first element in the vector. Cannot be used to modify the element.
@param vec A pointer to the vector from which to get the element.
@return A constant pointer to the first element in the vector, or NULL if the vector is NULL or empty.
@attention The returned pointer is a generic. You should cast it to the appropriate type before using it. The pointer will become invalid if the vector is resized or destroyed.
*/
static inline const void* vector_front_const(const vector_t* vec) {
if (!vec || vec->size == 0) {
return NULL; // Invalid vector or empty vector
}
return vec->data;
}
/*
@brief Gets a pointer to the last element in the vector.
@param vec A pointer to the vector from which to get the element.
@return A pointer to the last element in the vector, or NULL if the vector is NULL or empty.
@attention The returned pointer is a generic. You should cast it to the appropriate type before using it. The pointer will become invalid if the vector is resized or destroyed.
*/
static inline void* vector_back(vector_t* vec) {
if (!vec || vec->size == 0) {
return NULL; // Invalid vector or empty vector
}
return (char*)vec->data + ((vec->size - 1) * vec->element_size);
}
/*
@brief Gets a const pointer to the last element in the vector. Cannot be used to modify the element.
@param vec A pointer to the vector from which to get the element.
@return A constant pointer to the last element in the vector, or NULL if the vector is NULL or empty.
@attention The returned pointer is a generic. You should cast it to the appropriate type before using it. The pointer will become invalid if the vector is resized or destroyed.
*/
static inline const void* vector_back_const(const vector_t* vec) {
if (!vec || vec->size == 0) {
return NULL; // Invalid vector or empty vector
}
return (const char*)vec->data + ((vec->size - 1) * vec->element_size);
}
/*
@brief Gets a pointer to the element at the specified index in the vector.
@param vec A pointer to the vector from which to get the element.
@param index The index of the element to get.
@return A pointer to the element at the specified index, or NULL if the vector is NULL or the index is out of bounds.
@attention The returned pointer is a generic. You should cast it to the appropriate type before using it. The pointer will become invalid if the vector is resized or destroyed.
@attention For most read-only operations, consider using vector_get_const() instead, which returns a const pointer to the element.
*/
static inline void* vector_get(vector_t* vec, size_t index) {
if (!vec || index >= vec->size) {
return NULL; // Invalid vector or index out of bounds
}
return (char*)vec->data + (index * vec->element_size);
}
/*
@brief Gets a constant pointer to the element at the specified index in the vector. Cannot be used to modify the element.
@param vec A pointer to the vector from which to get the element.
@param index The index of the element to get.
@return A constant pointer to the element at the specified index, or NULL if the vector is NULL or the index is out of bounds.
@attention The returned pointer is a generic. You should cast it to the appropriate type before using it. The pointer will become invalid if the vector is resized or destroyed.
*/
static inline const void* vector_get_const(const vector_t* vec, size_t index) {
if (!vec || index >= vec->size) {
return NULL; // Invalid vector or index out of bounds
}
return (const char*)vec->data + (index * vec->element_size);
}
/*
@brief Gets the size of the vector.
@param vec A pointer to the vector whose size is to be retrieved.
@return The size of the vector, or 0 if the vector is NULL.
*/
static inline size_t vector_size(const vector_t* vec) {
if (!vec) {
return 0; // Invalid vector
}
return vec->size;
}
/*
@brief Gets the capacity of the vector.
@param vec A pointer to the vector whose capacity is to be retrieved.
@return The capacity of the vector, or 0 if the vector is NULL.
*/
static inline size_t vector_capacity(const vector_t* vec) {
if (!vec) {
return 0; // Invalid vector
}
return vec->capacity;
}
/*
@brief Gets the size of each element in the vector.
@param vec A pointer to the vector whose element size is to be retrieved.
@return The size of each element in the vector, or 0 if the vector is NULL.
*/
static inline size_t vector_element_size(const vector_t* vec) {
if (!vec) {
return 0; // Invalid vector
}
return vec->element_size;
}
/*
@brief Gets a pointer to the underlying C array of the vector. Cannot modify the vector through this pointer.
@param vec A pointer to the vector whose underlying array is to be retrieved.
@return A pointer to the underlying C array, or NULL if the vector is NULL.
@attention The returned pointer is a generic. You should cast it to the appropriate type before using it. The pointer will become invalid if the vector is resized or destroyed.
*/
static inline const void* vector_as_c_array(const vector_t* vec) {
if (!vec) {
return NULL; // Invalid vector
}
return vec->data;
}
/*
@brief Gets a pointer to the underlying C array of the vector. Can be used to modify the vector's elements.
@param vec A pointer to the vector whose underlying array is to be retrieved.
@return A pointer to the underlying C array, or NULL if the vector is NULL.
@attention The returned pointer is a generic. You should cast it to the appropriate type before using it. The pointer will become invalid if the vector is resized or destroyed.
*/
static inline void* vector_as_c_array_mutable(vector_t* vec) {
if (!vec) {
return NULL; // Invalid vector
}
return vec->data;
}
/*
@brief Moves a vector to another vector, transferring ownership of the data. The destination vector will take ownership of the source vector's data, its element size and its destructor.
@param dest A pointer to the destination vector.
@param src A pointer to the pointer holding the source vector. It will be set to NULL.
@return 0 on success, -1 if the destination vector is NULL, the source pointer is NULL, the source vector is NULL, or both vectors share the same data pointer.
@attention After calling this function, the source vector will be freed (excluding data) and the source pointer will be set to NULL, so it cannot be used again.
@attention The destination vector's existing elements will be destroyed (using the destination's own destructor, if it has one) and its data freed. Ensure that you do not need the existing data before calling this function.
@attention Moving a vector onto itself is a no-op and reports success, leaving the vector untouched.
@attention Two vectors can only share a data pointer if a vector_t was duplicated by copying the struct, which is never valid. The move is refused rather than freeing a buffer the source still points at.
*/
static inline int vector_move(vector_t* dest, vector_t** src) {
if (!dest || !src || !*src) {
return -1; // Invalid vectors
}
if (dest == *src) {
return 0; // Moving to itself, no action needed
}
if (dest->data == (*src)->data) {
return -1; // Refuse to move a vector onto another vector that shares its data
// Realistically, this is unreachable via legal use, but... safety.
// Or something. You WILL trigger this if you do shallow copies. DO NOT!
}
// Destroy the destination vector's existing elements with its own destructor,
// then free its data, as it is about to be replaced
vector_clear(dest);
free(dest->data);
// Transfer ownership of the source vector's data to the destination vector
dest->size = (*src)->size;
dest->capacity = (*src)->capacity;
dest->element_size = (*src)->element_size;
dest->data = (*src)->data;
dest->destructor = (*src)->destructor; // The elements keep the cleanup they came with
// Reset the source vector to an empty state
(*src)->size = 0;
(*src)->capacity = 0;
(*src)->element_size = 0;
(*src)->data = NULL;
(*src)->destructor = NULL;
free(*src); // Free the source vector structure, but not its data (ownership transferred)
*src = NULL;
return 0; // Success
}
/*
@brief Creates a deep copy of the vector, including its data.
@param vec A pointer to the vector to be copied.
@return A pointer to the newly created deep copy of the vector, or NULL if allocation fails, if the input vector is NULL, or if the input vector has a destructor set.
@attention The returned vector must be destroyed with vector_destroy() to free its memory. Failing to do so will result in a memory leak.
@attention Vectors with a destructor cannot be deep copied and this function will return NULL for them. The elements are copied byte for byte, so any memory they own would end up owned by both vectors and freed twice. If you need to copy such a vector, do it by hand: create a new vector and push copies of the elements into it yourself.
*/
static inline vector_t* vector_deep_copy(const vector_t* vec) {
if (!vec) {
return NULL; // Invalid vector
}
if (vec->destructor) {
return NULL; // Refuse to byte-copy elements that own memory
}
vector_t* new_vec = vector_create(vec->element_size);
if (!new_vec) {
return NULL; // Allocation failed
}
int r = vector_reserve(new_vec, vec->capacity);
if (r != 0) {
vector_destroy(&new_vec);
return NULL; // Allocation failed
}
if (vec->size > 0) {
memcpy(new_vec->data, vec->data, vec->size * vec->element_size);
new_vec->size = vec->size;
} // vector_create() already sets size to 0 by default
return new_vec;
}
/*
@brief Destroys the vector and frees its memory.
@param vec A pointer to the pointer holding the vector to be destroyed. It will be set to NULL.
@return 0 on success, or if the vector was already NULL. -1 only if the pointer itself is NULL.
@attention After calling this function, the vector pointer should not be used again (It will be set to NULL). Accessing it after destruction will lead to undefined behavior.
@attention Only the pointer you pass in is set to NULL. Copies of that pointer held elsewhere are left dangling and must not be used.
@attention If stored elements own memory of their own, set a destructor with vector_set_destructor() to have it cleaned up here. Otherwise, the vector will only free the memory allocated for the data array and the vector structure itself, but not any dynamically allocated memory within the elements.
*/
static inline int vector_destroy(vector_t** vec) {
if (!vec) {
return -1; // Invalid pointer
}
if (!(*vec)) {
return 0; // Already NULL, nothing to destroy
}
vector_t* target = *vec;
if (target->destructor) {
for (size_t i = 0; i < target->size; ++i) {
void* element = (char*)target->data + (i * target->element_size);
target->destructor(element); // Call the destructor for each element
}
}
free(target->data);
free(target);
*vec = NULL; // Set the pointer to NULL to avoid dangling references
return 0; // Success
}
#endif // VECTOR_H
include/dynarr.h
-63
View File
@@ -1,63 +0,0 @@
#ifndef DYNARR_H
#define DYNARR_H
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#define DYNARR_MAX_CAPACITY ((size_t)0x7FFFFFFF)
typedef struct {
size_t size;
size_t elemSize;
size_t capacity;
void* data;
} DynArr;
// Do not use; Use DYNARR_CREATE macro instead.
DynArr* DynArr_create(size_t elemSize, size_t capacity);
// Reserve n blocks in arary; New size will be n, NOT size + n; Reserving less memory that current will fail, use prune instead.
void DynArr_reserve(DynArr* p, size_t n);
// Push data into a new block at the end of the array
void* DynArr_push_back(DynArr* p, void* value);
// Remove the last block in the array.
void DynArr_pop_back(DynArr* p);
// Remove first block from array.
void DynArr_pop_front(DynArr* p);
// Remove index from array. This moves all blocks after the index block.
void DynArr_remove(DynArr* p, size_t index);
// Erase the array. This will not free unused blocks.
void DynArr_erase(DynArr* p);
// Prune and free unused blocks. If pruning to zero, ensure to reserve after.
void DynArr_prune(DynArr* p);
// Get a pointer to a block by index
void* DynArr_at(DynArr* p, size_t index);
// Get the index by block pointer
size_t DynArr_at_ptr(DynArr* p, void* ptr);
// Get size
size_t DynArr_size(DynArr* p);
// Get element size
size_t DynArr_elemSize(DynArr* p);
// Get capacity
size_t DynArr_capacity(DynArr* p);
void DynArr_destroy(DynArr* p);
// Note: Make sure to not overread or overwrite
void* DynArr_c_arr(DynArr* p);
#define DYNARR_CREATE(T, initialCapacity) DynArr_create(sizeof(T), initialCapacity)
#endif
include/dynset.h
-20
View File
@@ -1,20 +0,0 @@
#ifndef DYNSET_H
#define DYNSET_H
#include <dynarr.h>
// Dynamic Set structure - basically DynArr with uniqueness enforced
typedef struct {
DynArr* arr;
} DynSet;
// Function prototypes
DynSet* DynSet_Create(size_t elemSize);
void DynSet_Destroy(DynSet* set);
int DynSet_Insert(DynSet* set, const void* element);
int DynSet_Contains(DynSet* set, const void* element);
size_t DynSet_Size(DynSet* set);
void* DynSet_Get(DynSet* set, size_t index);
void DynSet_Remove(DynSet* set, const void* element);
#endif
include/khash/khash.h
-1
View File
@@ -113,7 +113,6 @@ int main() {
* Added destructor * Added destructor
*/ */
#ifndef __AC_KHASH_H #ifndef __AC_KHASH_H
#define __AC_KHASH_H #define __AC_KHASH_H
include/nets/net_node.h
+53 -4
View File
@@ -10,24 +10,29 @@
#include <constants.h> #include <constants.h>
#include <packettype.h> #include <packettype.h>
#include <udpd/udpnode.h>
// Forward declaration - the discovery state is defined in nodediscovery.c (opaque here).
typedef struct node_discovery node_discovery_t;
#include <stddef.h> #include <stddef.h>
#include <dynarr.h> #include <dlibc/vector.h>
#include <dynset.h> #include <dlibc/set.h>
#include <pthread.h> #include <pthread.h>
#include <block/block.h> #include <block/block.h>
#include <block/chain.h> #include <block/chain.h>
#include <block/transaction.h> #include <block/transaction.h>
#include <stdatomic.h>
typedef struct { typedef struct {
tcp_server_t* server; tcp_server_t* server;
tcp_client_t outboundClients[MAX_CONS]; tcp_client_t outboundClients[MAX_CONS];
size_t outboundCount; size_t outboundCount;
// Dedup cache for recently seen block hashes (canonical 32-byte hash) // Dedup cache for recently seen block hashes (canonical 32-byte hash)
DynSet* seenBlocks; set_t* seenBlocks;
// Protects seenBlocks // Protects seenBlocks
pthread_mutex_t seenLock; pthread_mutex_t seenLock;
// Protects outboundClients snapshots and peerBlockHeight writes // Protects outboundClients snapshots and peerBlockHeight writes
@@ -38,8 +43,14 @@ typedef struct {
void* callbackUser; void* callbackUser;
// Maintenance thread for periodic tasks (orphan attach, pruning, metrics) // Maintenance thread for periodic tasks (orphan attach, pruning, metrics)
pthread_t maintenanceThread; pthread_t maintenanceThread;
volatile int maintenanceRunning; // Cross-thread stop flag: written by Node_Destroy on the main thread, read by the maintenance
// thread's loop condition. `volatile` stops the compiler hoisting the load but provides neither
// atomicity nor ordering, so this has to be a real atomic (and TSan rightly flagged it).
_Atomic int maintenanceRunning;
int maintenanceIntervalMs; int maintenanceIntervalMs;
// UDP ping/pong daemon (latency oracle) and peer discovery state
udp_node_t* udpNode;
node_discovery_t* discovery;
} net_node_t; } net_node_t;
net_node_t* Node_Create(); net_node_t* Node_Create();
@@ -61,6 +72,29 @@ int Node_BroadcastTransaction(net_node_t* node, signed_transaction_t* tx, tcp_co
// Helpers for outbound peer selection and block broadcast // Helpers for outbound peer selection and block broadcast
int Node_GetBestOutboundPeer(net_node_t* node, tcp_connection_t** outConn, uint64_t* outHeight); int Node_GetBestOutboundPeer(net_node_t* node, tcp_connection_t** outConn, uint64_t* outHeight);
/**
* Delivery receipts for windowed sync.
*
* A FETCH_BLOCK reply is handled on the peer's io thread and may legitimately never reach the
* chain: a block belonging to a competing branch is filed in the orphan pool instead. A sync loop
* that infers arrival from the chain growing therefore cannot tell "arrived but forked" from "lost
* in transit", so it re-requests until it times out. Against a peer on a fork that costs one full
* retry-and-timeout cycle for EVERY block, which is why syncing to a forked peer used to crawl.
*
* DUPLICATE is what makes a backwards fork walk terminate: it means we already hold exactly that
* block, so the two chains agree at that height and there is no reason to keep descending.
**/
typedef enum {
NODE_DELIVERY_APPENDED = 0, // joined our chain
NODE_DELIVERY_DUPLICATE = 1, // we already held this exact block -- common ground
NODE_DELIVERY_ORPHANED = 2, // belongs to a competing branch; now in the orphan pool
NODE_DELIVERY_REJECTED = 3 // failed validation
} node_delivery_status_t;
void Node_NoteBlockDelivered(uint64_t height, node_delivery_status_t status);
bool Node_TakeBlockDelivery(uint64_t height, node_delivery_status_t* outStatus);
void Node_ResetBlockDeliveries(void);
void Node_BroadcastChainRange(net_node_t* node, size_t startHeightInclusive, tcp_connection_t* sourceConn); void Node_BroadcastChainRange(net_node_t* node, size_t startHeightInclusive, tcp_connection_t* sourceConn);
// Callback logic // Callback logic
@@ -71,4 +105,19 @@ void Node_Client_OnConnect(tcp_connection_t* client);
void Node_Client_OnData(tcp_connection_t* client); void Node_Client_OnData(tcp_connection_t* client);
void Node_Client_OnDisconnect(tcp_connection_t* client); void Node_Client_OnDisconnect(tcp_connection_t* client);
void Node_GetClientList(net_node_t* node, tcp_connection_t** outClients, size_t* outCount);
// Computes a connection's peer listen endpoint (IP + advertised/dialed listen port) into *out.
// Outbound: the dialed peerAddr port already is the listen port. Inbound: uses peerListenPort.
// Returns non-zero on success (usable endpoint with a known, non-zero port), zero otherwise.
int Node_ConnListenEndpoint(const tcp_connection_t* conn, struct sockaddr_storage* out);
// Returns the node identity advertised by a connection's peer, or 0 if it is not known yet.
uint64_t Node_ConnPeerNodeId(const tcp_connection_t* conn);
// Fills outEndpoints with the listen endpoints of all current connections (inbound + outbound),
// deduped by IP+port, and outNodeIds (optional, may be NULL) with the matching peer identities.
// Returns the number of endpoints written (<= maxOut).
size_t Node_GetPeerEndpoints(net_node_t* node, struct sockaddr_storage* outEndpoints, uint64_t* outNodeIds, size_t maxOut);
#endif #endif
include/nets/nodediscovery.h
+45
View File
@@ -0,0 +1,45 @@
#ifndef NODEDISCOVERY_H
#define NODEDISCOVERY_H
#include <nets/net_node.h>
#include <udpd/udpnode.h>
// Create/destroy the peer-discovery state. Owns the known-peer table and its lock.
node_discovery_t* NodeDiscovery_Create(net_node_t* node, udp_node_t* udpNode);
void NodeDiscovery_Destroy(node_discovery_t* disc);
// Periodic tick (driven by the node maintenance thread): seed currently-connected peers,
// UDP-ping newly-learned ones, query a couple of connected peers for more, and connect to
// the reachable peers with the lowest ping until we reach the target connection count.
void NodeDiscovery_Iterate(node_discovery_t* disc);
// UDP latency callbacks (forwarded from the udp node via net_node thunks).
void NodeDiscovery_OnPong(node_discovery_t* disc, const struct sockaddr_storage* from, uint64_t nonce, uint64_t rttMs);
void NodeDiscovery_OnPingTimeout(node_discovery_t* disc, const struct sockaddr_storage* dest, uint64_t nonce);
// TCP peer-exchange handlers (called from the net_node packet dispatch).
// Build a PEERS response (a sample of our peers, excluding the requester) and send it over fromConn.
void NodeDiscovery_OnGetPeers(node_discovery_t* disc, tcp_connection_t* fromConn);
// Decode a received PEERS payload and fold a couple of its endpoints into the known-peer table.
void NodeDiscovery_OnPeersReceived(node_discovery_t* disc, tcp_connection_t* fromConn, const unsigned char* payload, size_t payloadLen);
// Strike a peer (by its listen endpoint) from the known-peer table. Called when a peer becomes
// logically disconnected (no remaining connection to it).
void NodeDiscovery_RemovePeer(node_discovery_t* disc, const struct sockaddr_storage* endpoint);
// Record the node identity behind an endpoint (learned from a completed HELLO/ACK_HELLO). Entries
// carrying an identity we are already connected to are skipped by the connect picker, which is what
// stops a multi-homed peer from being dialed once per address it is reachable on.
void NodeDiscovery_NoteIdentity(node_discovery_t* disc, const struct sockaddr_storage* endpoint, uint64_t nodeId);
// Mark an endpoint as one of our own, permanently. Self endpoints are never added to the known-peer
// table, never pinged and never dialed. Seeded from the local interface addresses at creation and
// extended whenever a handshake turns out to come from ourselves.
void NodeDiscovery_MarkSelfEndpoint(node_discovery_t* disc, const struct sockaddr_storage* endpoint);
// Returns non-zero if the endpoint is known to be one of our own.
int NodeDiscovery_IsSelfEndpoint(node_discovery_t* disc, const struct sockaddr_storage* endpoint);
// Dump the known-peer table to stdout (for the CLI `peers` command).
void NodeDiscovery_PrintPeers(node_discovery_t* disc);
#endif
include/nets/orphan_pool.h
+25 -3
View File
@@ -2,6 +2,7 @@
#define ORPHAN_POOL_H #define ORPHAN_POOL_H
#include <stdint.h> #include <stdint.h>
#include <stdbool.h>
#include <block/block.h> #include <block/block.h>
#include <block/chain.h> #include <block/chain.h>
@@ -10,11 +11,32 @@ void OrphanPool_Init(void);
void OrphanPool_Destroy(void); void OrphanPool_Destroy(void);
// Insert an orphan block into the pool. Ownership of `block` is transferred to the pool. // Insert an orphan block into the pool. Ownership of `block` is transferred to the pool.
// `height` is the block number from the header. // `height` is the block number from the header. `observedAtTipHeight` is the local chain tip
void OrphanPool_Insert(block_t* block, uint64_t height); // height at the moment the block arrived; it is stamped once and drives the Horizen reorg
// penalty, so it must never be re-derived from a later tip.
// Duplicates (same block hash) are rejected and the block is destroyed.
void OrphanPool_Insert(block_t* block, uint64_t height, uint64_t observedAtTipHeight);
// Attempt to attach any orphans whose parents now exist in `chain`. // Attempt to attach any orphans whose parents now exist in `chain`, and to adopt a competing
// branch when one is heavier and has served its reorg penalty.
// Returns the number of blocks successfully attached. // Returns the number of blocks successfully attached.
size_t OrphanPool_AttemptAttach(blockchain_t* chain); size_t OrphanPool_AttemptAttach(blockchain_t* chain);
/**
* As OrphanPool_AttemptAttach, but skips the reorg delay penalty when `bypassPenalty` is set.
*
* Reserved for an explicit operator action (`sync force`). The penalty is served by local chain
* growth, so a node that is neither mining nor stale enough to count as catching up can never
* clear it by itself; this is the manual way out for an operator who knows their branch is the
* wrong one. Work comparison and linkage still apply, so it cannot adopt a lighter branch, and
* nothing a peer sends can reach it.
**/
size_t OrphanPool_AttemptAttachForced(blockchain_t* chain, bool bypassPenalty);
// True if a block with this hash is already pooled.
bool OrphanPool_Contains(const uint8_t blockHash[32]);
// Number of pooled orphans (diagnostics).
size_t OrphanPool_Size(void);
#endif #endif
include/numgen.h
+5
View File
@@ -11,4 +11,9 @@ uint16_t random_two_byte(void);
uint32_t random_four_byte(void); uint32_t random_four_byte(void);
uint64_t random_eight_byte(void); uint64_t random_eight_byte(void);
// Draws from the OS entropy pool instead of the srand()-seeded generator, which repeats across
// processes started within the same second. Use this wherever a value must be unique between nodes
// (e.g. the node identity). Never returns 0.
uint64_t random_secure_eight_byte(void);
#endif #endif
include/packettype.h
+3 -1
View File
@@ -14,7 +14,9 @@ typedef enum {
PACKET_TYPE_BROADCAST_TX = 7, // Here's a new transaction I want to share with the network PACKET_TYPE_BROADCAST_TX = 7, // Here's a new transaction I want to share with the network
PACKET_TYPE_ACK_TX = 8, // I have received your transaction, here's what I did with it (response to broadcast) PACKET_TYPE_ACK_TX = 8, // I have received your transaction, here's what I did with it (response to broadcast)
PACKET_TYPE_ERROR = 9, // Something went wrong with the packet you sent me, here's an error message (can be response to any packet) PACKET_TYPE_ERROR = 9, // Something went wrong with the packet you sent me, here's an error message (can be response to any packet)
PACKET_TYPE_MAX = 10 PACKET_TYPE_GET_PEERS = 10, // Who are your peers? Send me a few of them so I can discover more of the network
PACKET_TYPE_PEERS = 11, // Here are some of my peers' listen endpoints (response to GET_PEERS)
PACKET_TYPE_MAX = 12
} packet_type_t; } packet_type_t;
static inline int PacketType_IsValid(uint8_t packetType) { static inline int PacketType_IsValid(uint8_t packetType) {
include/runtime_state.h
+15 -11
View File
@@ -8,18 +8,22 @@
#include <block/chain.h> #include <block/chain.h>
#include <pthread.h> #include <pthread.h>
extern uint64_t currentBlockHeight; extern uint64_t g_currentBlockHeight;
extern blockchain_t* currentChain; extern blockchain_t* g_currentChain;
extern uint256_t currentSupply; extern uint256_t g_currentSupply;
extern uint64_t currentReward; extern uint64_t g_currentReward;
extern uint32_t difficultyTarget; extern uint32_t g_difficultyTarget;
extern const char* chainDataDir; extern const char* g_chainDataDir;
extern unsigned short listenPort; extern unsigned short g_listenPort;
extern bool echoPeersEnabled; extern bool g_echoPeersEnabled;
extern bool forceOrphanReorgEnabled; extern bool g_forceOrphanReorgEnabled;
// Random per-run identity of this node, advertised in HELLO/ACK_HELLO. A host can be reachable
// under many addresses (especially over IPv6), so an (ip, port) endpoint is not a peer identity:
// this nonce is what lets us recognise our own connections and a peer we already talk to.
extern uint64_t g_localNodeId;
// Global synchronization primitives for runtime state // Global synchronization primitives for runtime state
extern pthread_rwlock_t chainLock; // protects chain structure and related mutations extern pthread_rwlock_t g_chainLock; // protects chain structure and related mutations
extern pthread_mutex_t balanceSheetLock; // protects balance sheet map extern pthread_mutex_t g_balanceSheetLock; // protects balance sheet map
#endif #endif
include/tcpd/tcpconnection.h
+20
View File
@@ -3,6 +3,7 @@
#include <arpa/inet.h> #include <arpa/inet.h>
#include <pthread.h> #include <pthread.h>
#include <stdatomic.h>
#include <stdbool.h> #include <stdbool.h>
#include <stddef.h> #include <stddef.h>
#include <stdint.h> #include <stdint.h>
@@ -26,6 +27,15 @@ struct tcp_connection_t {
uint32_t connectionId; uint32_t connectionId;
tcp_connection_role_t role; tcp_connection_role_t role;
// Peer's advertised TCP/UDP listen port (learned from HELLO/ACK_HELLO). 0 until known.
// For OUTBOUND connections the peerAddr port already is the listen port; this matters for INBOUND peers.
uint16_t peerListenPort;
// Peer's advertised node identity (learned from HELLO/ACK_HELLO). 0 until known / peer too old
// to advertise one. Unlike the peer address, this is stable across all of a multi-homed peer's
// endpoints, so it is what identifies the node behind this connection.
uint64_t peerNodeId;
pthread_t ioThread; pthread_t ioThread;
pthread_mutex_t sendLock; pthread_mutex_t sendLock;
pthread_mutex_t stateLock; pthread_mutex_t stateLock;
@@ -33,6 +43,11 @@ struct tcp_connection_t {
bool closing; bool closing;
bool disconnectedNotified; bool disconnectedNotified;
// Non-zero while another thread holds a raw pointer to this connection taken from a
// lock-protected snapshot and used after releasing the lock. The reaper must not free a
// pinned connection. See TcpConnection_Pin/Unpin.
atomic_int pinCount;
unsigned char* dataBuf; unsigned char* dataBuf;
size_t dataBufLen; size_t dataBufLen;
size_t dataBufCap; size_t dataBufCap;
@@ -71,4 +86,9 @@ void TcpConnection_RequestClose(tcp_connection_t* conn);
void TcpConnection_MarkDisconnectNotified(tcp_connection_t* conn); void TcpConnection_MarkDisconnectNotified(tcp_connection_t* conn);
bool TcpConnection_IsDisconnectNotified(tcp_connection_t* conn); bool TcpConnection_IsDisconnectNotified(tcp_connection_t* conn);
// Pin/unpin a connection so a background reaper won't free it while a caller still holds a raw
// pointer to it (e.g. across a blocking operation after releasing the collection lock).
void TcpConnection_Pin(tcp_connection_t* conn);
void TcpConnection_Unpin(tcp_connection_t* conn);
#endif #endif
include/tcpd/tcpserver.h
+4 -1
View File
@@ -7,12 +7,15 @@
#include <constants.h> #include <constants.h>
#include <tcpd/tcpconnection.h> #include <tcpd/tcpconnection.h>
#include <stdatomic.h>
typedef struct { typedef struct {
int sockFd; // IPv6 listening socket (-1 if IPv6 unavailable) int sockFd; // IPv6 listening socket (-1 if IPv6 unavailable)
int sockFdV4; // IPv4 listening socket (-1 on bind failure) int sockFdV4; // IPv4 listening socket (-1 on bind failure)
int opt; int opt;
int isRunning; // Cross-thread stop flag: cleared by TcpServer_Stop, read by both accept threads and by
// exiting client threads. Must be atomic, not a plain int.
_Atomic int isRunning;
void* owner; void* owner;
// Called before the client thread runs // Called before the client thread runs
include/txmempool.h
+30 -2
View File
@@ -2,12 +2,21 @@
#define TXMEMPOOL_H #define TXMEMPOOL_H
#include <block/transaction.h> #include <block/transaction.h>
#include <khash/khash.h> #include <khash.h>
#include <utils.h> #include <utils.h>
#include <uint256.h> #include <uint256.h>
// See balance_sheet.h: khash's macro expansion is not -Wconversion clean, and
// -isystem does not cover code expanded from a system header's macros.
#if defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wconversion"
#endif
KHASH_INIT(tx_mempool_map_m, key32_t, signed_transaction_t, 1, hash_key32, eq_key32) KHASH_INIT(tx_mempool_map_m, key32_t, signed_transaction_t, 1, hash_key32, eq_key32)
extern khash_t(tx_mempool_map_m)* txMempool; #if defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic pop
#endif
extern khash_t(tx_mempool_map_m)* g_txMempool;
void TxMempool_Init(); void TxMempool_Init();
// Assumed that the transation was confirmed to be valid // Assumed that the transation was confirmed to be valid
@@ -17,6 +26,25 @@ bool TxMempool_Snapshot(signed_transaction_t** outTxs, size_t* outCount);
void TxMempool_Print(); void TxMempool_Print();
// Remove a transaction from the mempool by its hash. Returns true if removed. // Remove a transaction from the mempool by its hash. Returns true if removed.
bool TxMempool_Remove(const uint8_t* txHash); bool TxMempool_Remove(const uint8_t* txHash);
/**
* Admission policy: should this transaction be held and relayed?
*
* LOCAL POLICY, NOT CONSENSUS. A block containing a transaction this rejects is still accepted --
* see TX_MAX_FUTURE_DRIFT_MS / TX_EXPIRY_MS in constants.h for why the two are kept apart.
*
* Both bounds are measured against the node's own clock, NOT against the chain tip's timestamp.
* Measuring "future" against the last block assumes blocks keep arriving: on a quiet chain the tip
* can be hours old, and an honest transaction created right now would look hours ahead of it and be
* refused. Sending would become impossible exactly when the chain is idle.
*
* Deliberately NOT applied when a rollback returns transactions to the pool: those were already in
* the chain, so they are legitimate by definition and must not be dropped for looking old.
**/
bool TxMempool_PolicyAccepts(const signed_transaction_t* tx, uint64_t nowMs);
// Drop transactions older than TX_EXPIRY_MS. Returns how many were removed.
size_t TxMempool_PruneExpired(uint64_t nowMs);
void TxMempool_Destroy(); void TxMempool_Destroy();
#endif #endif
include/udpd/udpnode.h
+59
View File
@@ -0,0 +1,59 @@
#ifndef UDP_NODE_H
#define UDP_NODE_H
#include <stdint.h>
#include <stdbool.h>
#include <pthread.h>
#include <netinet/in.h>
#include <udpd/udppackettype.h>
#include <stdatomic.h>
#define UDP_LISTEN_PORT 9393
#define UDP_PING_RETRY_INTERVAL_MS 1000
#define UDP_PING_MAX_RETRIES 3
#define UDP_MAX_PENDING_PINGS 64
typedef struct {
uint64_t nonce;
struct sockaddr_storage dest;
uint64_t lastSentMs;
int retries;
bool active;
} pending_ping_t;
typedef struct udp_node {
int sockFd; // AF_INET6, IPV6_V6ONLY=1
int sockFdV4; // AF_INET
// Cross-thread stop flag: cleared by UdpNode_Stop, read by the recv and retry thread loops.
// See the note on net_node_t.maintenanceRunning -- volatile is not a substitute for atomic.
_Atomic int isRunning;
pthread_t recvThreadV6;
pthread_t recvThreadV4;
pthread_t retryThread;
pending_ping_t pendingPings[UDP_MAX_PENDING_PINGS];
pthread_mutex_t pingsMutex;
void (*on_pong)(struct udp_node* node,
const struct sockaddr_storage* from,
uint64_t nonce, int protoVersion, uint64_t rttMs, void* user);
void (*on_ping_timeout)(struct udp_node* node,
const struct sockaddr_storage* dest,
uint64_t nonce, void* user);
void* callbackUser;
} udp_node_t;
int UdpNode_Init(udp_node_t* node, uint16_t port);
void UdpNode_SetCallbacks(udp_node_t* node,
void (*on_pong)(udp_node_t*, const struct sockaddr_storage*, uint64_t, int, uint64_t, void*),
void (*on_ping_timeout)(udp_node_t*, const struct sockaddr_storage*, uint64_t, void*),
void* user);
int UdpNode_Start(udp_node_t* node);
void UdpNode_Stop(udp_node_t* node);
void UdpNode_Destroy(udp_node_t* node);
int UdpNode_SendPing(udp_node_t* node, const struct sockaddr_storage* dest);
#endif
include/udpd/udppackettype.h
+14
View File
@@ -0,0 +1,14 @@
#ifndef UDP_PACKET_TYPE_H
#define UDP_PACKET_TYPE_H
typedef enum {
UDP_PACKET_TYPE_NONE = 0,
UDP_PACKET_TYPE_PING = 1,
UDP_PACKET_TYPE_PONG = 2,
} udp_packet_type_t;
// Wire sizes in bytes
#define UDP_PING_WIRE_SIZE 9 // 1 (type) + 8 (nonce)
#define UDP_PONG_WIRE_SIZE 13 // 1 (type) + 8 (nonce) + 4 (proto_version)
#endif
include/uint256.h
+66
View File
@@ -114,6 +114,72 @@ static inline int uint256_cmp(const uint256_t* a, const uint256_t* b) {
return 0; return 0;
} }
static inline bool uint256_is_zero(const uint256_t* a) {
return a && a->limbs[0] == 0 && a->limbs[1] == 0 && a->limbs[2] == 0 && a->limbs[3] == 0;
}
/**
* Builds a uint256 from 32 big-endian bytes, the layout used by hashes and by decoded
* difficulty targets (see DecodeCompactTarget).
**/
static inline uint256_t uint256_from_be_bytes(const uint8_t bytes[32]) {
uint256_t res = {{0, 0, 0, 0}};
if (!bytes) {
return res;
}
for (int limb = 0; limb < 4; ++limb) {
// limbs[0] is the least significant, so it holds the LAST eight bytes.
const uint8_t* src = bytes + (3 - limb) * 8;
uint64_t value = 0;
for (int b = 0; b < 8; ++b) {
value = (value << 8) | (uint64_t)src[b];
}
res.limbs[limb] = value;
}
return res;
}
static inline void uint256_bitwise_not(uint256_t* a) {
if (!a) {
return;
}
for (int i = 0; i < 4; ++i) {
a->limbs[i] = ~a->limbs[i];
}
}
/**
* Unsigned 256-bit division by restoring binary long division.
* Returns false (leaving *outQuotient untouched) when dividing by zero.
**/
static inline bool uint256_divide(const uint256_t* numerator, const uint256_t* denominator, uint256_t* outQuotient) {
if (!numerator || !denominator || !outQuotient || uint256_is_zero(denominator)) {
return false;
}
uint256_t quotient = uint256_from_u64(0);
uint256_t remainder = uint256_from_u64(0);
for (int bit = 255; bit >= 0; --bit) {
// remainder = (remainder << 1) | bit_of_numerator
for (int i = 3; i > 0; --i) {
remainder.limbs[i] = (remainder.limbs[i] << 1) | (remainder.limbs[i - 1] >> 63);
}
remainder.limbs[0] <<= 1;
remainder.limbs[0] |= (numerator->limbs[bit / 64] >> (bit % 64)) & 1ULL;
if (uint256_cmp(&remainder, denominator) >= 0) {
(void)uint256_subtract(&remainder, denominator);
quotient.limbs[bit / 64] |= (1ULL << (bit % 64));
}
}
*outQuotient = quotient;
return true;
}
static inline void uint256_serialize(const uint256_t* value, char* out) { static inline void uint256_serialize(const uint256_t* value, char* out) {
if (!value || !out) { if (!value || !out) {
return; return;
include/utils.h
+35 -38
View File
@@ -9,11 +9,16 @@
#include <crypto/crypto.h> #include <crypto/crypto.h>
#include <uint256.h> #include <uint256.h>
#include <time.h> #include <time.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
typedef struct { typedef struct {
uint8_t bytes[32]; uint8_t bytes[32];
} key32_t; } key32_t;
#define PROTO_VERSION 1
static inline uint32_t hash_key32(key32_t k) { static inline uint32_t hash_key32(key32_t k) {
uint32_t hash = 2166136261u; uint32_t hash = 2166136261u;
for (int i = 0; i < 32; i++) { for (int i = 0; i < 32; i++) {
@@ -33,7 +38,7 @@ static inline uint64_t get_current_time_ms(void) {
return 0; // Handle error return 0; // Handle error
} }
// Convert seconds to milliseconds and add nanoseconds converted to milliseconds // Convert seconds to milliseconds and add nanoseconds converted to milliseconds
return (spec.tv_sec * 1000) + (spec.tv_nsec / 1000000); return ((uint64_t)spec.tv_sec * 1000ULL) + ((uint64_t)spec.tv_nsec / 1000000ULL);
} }
static inline void sleep_for_microseconds(uint64_t microseconds) { static inline void sleep_for_microseconds(uint64_t microseconds) {
@@ -250,53 +255,45 @@ static inline bool ParseHexAddress32(const char* in, uint8_t outAddress[32]) {
} }
static inline bool IsValidIPv4(const char* ip) { static inline bool IsValidIPv4(const char* ip) {
struct addrinfo hints, *res;
int status;
if (!ip || *ip == '\0') { if (!ip || *ip == '\0') {
return false; return false;
} }
int octetCount = 0; memset(&hints, 0, sizeof hints);
const char* p = ip; hints.ai_family = AF_INET; // Only IPv4
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_NUMERICHOST; // Only numeric addresses, no DNS lookups
while (*p != '\0') { status = getaddrinfo(ip, NULL, &hints, &res);
if (octetCount >= 4) { if (status == 0) {
freeaddrinfo(res);
return true;
}
return false;
}
static inline bool IsValidIPv6(const char* ip) {
struct addrinfo hints, *res;
int status;
if (!ip || *ip == '\0') {
return false; return false;
} }
if (*p < '0' || *p > '9') { memset(&hints, 0, sizeof hints);
return false; hints.ai_family = AF_INET6; // Only IPv6
} hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_NUMERICHOST; // Only numeric addresses, no DNS lookups
unsigned int value = 0; status = getaddrinfo(ip, NULL, &hints, &res);
int digits = 0; if (status == 0) {
while (*p >= '0' && *p <= '9') { freeaddrinfo(res);
value = (value * 10u) + (unsigned int)(*p - '0'); return true;
if (value > 255u) { }
return false; return false;
}
++digits;
if (digits > 3) {
return false;
}
++p;
}
if (digits == 0) {
return false;
}
++octetCount;
if (octetCount < 4) {
if (*p != '.') {
return false;
}
++p;
if (*p == '\0') {
return false;
}
}
}
return octetCount == 4;
} }
static inline void Uint256ToDecimal(const uint256_t* value, char* out, size_t outSize) { static inline void Uint256ToDecimal(const uint256_t* value, char* out, size_t outSize) {
src/autolykos2/autolykos2.c
+7 -38
View File
@@ -10,7 +10,7 @@
#include <unistd.h> #include <unistd.h>
#endif #endif
uint64_t autolykos2_sleepBetweenHashOperationsMicroseconds = 0; uint64_t g_autolykos2_sleepBetweenHashOperationsMicroseconds = 0;
typedef struct { typedef struct {
uint8_t* buf; uint8_t* buf;
@@ -249,12 +249,12 @@ static bool Autolykos2_HashCore(
bool ok = Blake2b_Hash(finalInput, sizeof(finalInput), outHash, 32); bool ok = Blake2b_Hash(finalInput, sizeof(finalInput), outHash, 32);
// Throttle between hash operations if configured (applies to all hash paths). // Throttle between hash operations if configured (applies to all hash paths).
if (autolykos2_sleepBetweenHashOperationsMicroseconds > 0) { if (g_autolykos2_sleepBetweenHashOperationsMicroseconds > 0) {
#if defined(_WIN32) || defined(_WIN64) #if defined(_WIN32) || defined(_WIN64)
DWORD ms = (DWORD)((autolykos2_sleepBetweenHashOperationsMicroseconds + 999) / 1000); DWORD ms = (DWORD)((g_autolykos2_sleepBetweenHashOperationsMicroseconds + 999) / 1000);
Sleep(ms); Sleep(ms);
#else #else
sleep_for_microseconds(autolykos2_sleepBetweenHashOperationsMicroseconds); sleep_for_microseconds(g_autolykos2_sleepBetweenHashOperationsMicroseconds);
#endif #endif
} }
@@ -409,37 +409,6 @@ bool Autolykos2_Hash(
); );
} }
bool Autolykos2_LightHash(const uint8_t* seed, blockchain_t* chain, uint64_t nonce, uint8_t* out) {
if (!seed || !chain || !out) {
return false;
}
const uint64_t height = (uint64_t)Chain_Size(chain);
const size_t dagBytes = CalculateTargetDAGSize(chain);
if (dagBytes < 32 || (dagBytes % 32) != 0) {
return false;
}
const size_t laneCount64 = dagBytes / 32u;
if (laneCount64 == 0 || laneCount64 > UINT32_MAX) {
return false;
}
// Light path derives the needed DAG lanes from seed on-demand, no large DAG allocation required.
return Autolykos2_HashCore(
seed,
seed,
seed,
32,
nonce,
height,
(uint32_t)laneCount64,
NULL,
false,
out
);
}
bool Autolykos2_LightHashAtHeight( bool Autolykos2_LightHashAtHeight(
const uint8_t seed32[32], const uint8_t seed32[32],
const uint8_t* message, const uint8_t* message,
@@ -490,8 +459,8 @@ bool Autolykos2_CheckTarget(
} }
#ifdef SKALACOIN_AUTOLYKOS2_REF_AVAILABLE #ifdef SKALACOIN_AUTOLYKOS2_REF_AVAILABLE
if (ctx->backend) { if (ctx->backend && height <= UINT32_MAX) {
const bool ok = skalacoin_autolykos2_ref_check_target(ctx->backend, message32, nonce, height, target32); const bool ok = skalacoin_autolykos2_ref_check_target(ctx->backend, message32, nonce, (uint32_t)height, target32);
if (Autolykos2_Hash(ctx, message32, 32, nonce, height, outHash)) { if (Autolykos2_Hash(ctx, message32, 32, nonce, height, outHash)) {
return ok; return ok;
} }
@@ -531,5 +500,5 @@ bool Autolykos2_FindNonceSingleCore(
} }
void Autolykos2_SetSleepBetweenHashOperations(uint64_t sleepMicroseconds) { void Autolykos2_SetSleepBetweenHashOperations(uint64_t sleepMicroseconds) {
autolykos2_sleepBetweenHashOperationsMicroseconds = sleepMicroseconds; g_autolykos2_sleepBetweenHashOperationsMicroseconds = sleepMicroseconds;
} }
src/balance_sheet.c
+24 -24
View File
@@ -1,7 +1,7 @@
#include <balance_sheet.h> #include <balance_sheet.h>
#include <pthread.h> #include <pthread.h>
khash_t(balance_sheet_map_m)* sheetMap = NULL; khash_t(balance_sheet_map_m)* g_sheetMap = NULL;
static pthread_mutex_t g_sheetLock; static pthread_mutex_t g_sheetLock;
static bool BalanceSheet_GetSimEntry( static bool BalanceSheet_GetSimEntry(
@@ -22,7 +22,7 @@ static bool BalanceSheet_GetSimEntry(
return true; return true;
} }
if (BalanceSheet_Lookup((uint8_t*)address, out)) { if (BalanceSheet_Lookup(address, out)) {
int ret = 0; int ret = 0;
k = kh_put(balance_sheet_map_m, simMap, key, &ret); k = kh_put(balance_sheet_map_m, simMap, key, &ret);
if (k == kh_end(simMap)) { if (k == kh_end(simMap)) {
@@ -139,7 +139,7 @@ static bool BalanceSheet_ApplyCandidateTransaction(
} }
static int BalanceSheet_InsertLocked(balance_sheet_entry_t entry) { static int BalanceSheet_InsertLocked(balance_sheet_entry_t entry) {
if (!sheetMap) { if (!g_sheetMap) {
return -1; return -1;
} }
@@ -147,22 +147,22 @@ static int BalanceSheet_InsertLocked(balance_sheet_entry_t entry) {
memcpy(key.bytes, entry.address, 32); memcpy(key.bytes, entry.address, 32);
int ret = 0; int ret = 0;
khiter_t k = kh_put(balance_sheet_map_m, sheetMap, key, &ret); khiter_t k = kh_put(balance_sheet_map_m, g_sheetMap, key, &ret);
if (k == kh_end(sheetMap)) { if (k == kh_end(g_sheetMap)) {
return -1; return -1;
} }
kh_value(sheetMap, k) = entry; kh_value(g_sheetMap, k) = entry;
return ret; return ret;
} }
void BalanceSheet_Init() { void BalanceSheet_Init() {
sheetMap = kh_init(balance_sheet_map_m); g_sheetMap = kh_init(balance_sheet_map_m);
pthread_mutex_init(&g_sheetLock, NULL); pthread_mutex_init(&g_sheetLock, NULL);
} }
int BalanceSheet_Insert(balance_sheet_entry_t entry) { int BalanceSheet_Insert(balance_sheet_entry_t entry) {
if (!sheetMap) { return -1; } if (!g_sheetMap) { return -1; }
pthread_mutex_lock(&g_sheetLock); pthread_mutex_lock(&g_sheetLock);
int ret = BalanceSheet_InsertLocked(entry); int ret = BalanceSheet_InsertLocked(entry);
@@ -170,16 +170,16 @@ int BalanceSheet_Insert(balance_sheet_entry_t entry) {
return ret; return ret;
} }
bool BalanceSheet_Lookup(uint8_t* address, balance_sheet_entry_t* out) { bool BalanceSheet_Lookup(const uint8_t* address, balance_sheet_entry_t* out) {
if (!address || !out) { return false; } if (!address || !out) { return false; }
pthread_mutex_lock(&g_sheetLock); pthread_mutex_lock(&g_sheetLock);
key32_t key; key32_t key;
memcpy(key.bytes, address, 32); memcpy(key.bytes, address, 32);
khiter_t k = kh_get(balance_sheet_map_m, sheetMap, key); khiter_t k = kh_get(balance_sheet_map_m, g_sheetMap, key);
if (k != kh_end(sheetMap)) { if (k != kh_end(g_sheetMap)) {
balance_sheet_entry_t entry = kh_value(sheetMap, k); balance_sheet_entry_t entry = kh_value(g_sheetMap, k);
memcpy(out, &entry, sizeof(balance_sheet_entry_t)); memcpy(out, &entry, sizeof(balance_sheet_entry_t));
pthread_mutex_unlock(&g_sheetLock); pthread_mutex_unlock(&g_sheetLock);
return true; return true;
@@ -190,7 +190,7 @@ bool BalanceSheet_Lookup(uint8_t* address, balance_sheet_entry_t* out) {
} }
bool BalanceSheet_SaveToFile(const char* outPath) { bool BalanceSheet_SaveToFile(const char* outPath) {
if (!sheetMap) { return false; } if (!g_sheetMap) { return false; }
pthread_mutex_lock(&g_sheetLock); pthread_mutex_lock(&g_sheetLock);
char outFile[512]; char outFile[512];
@@ -203,9 +203,9 @@ bool BalanceSheet_SaveToFile(const char* outPath) {
} }
khiter_t k; khiter_t k;
for (k = kh_begin(sheetMap); k != kh_end(sheetMap); ++k) { for (k = kh_begin(g_sheetMap); k != kh_end(g_sheetMap); ++k) {
if (kh_exist(sheetMap, k)) { if (kh_exist(g_sheetMap, k)) {
balance_sheet_entry_t entry = kh_val(sheetMap, k); balance_sheet_entry_t entry = kh_val(g_sheetMap, k);
if (fwrite(&entry, sizeof(balance_sheet_entry_t), 1, file) != 1) { if (fwrite(&entry, sizeof(balance_sheet_entry_t), 1, file) != 1) {
fclose(file); fclose(file);
pthread_mutex_unlock(&g_sheetLock); pthread_mutex_unlock(&g_sheetLock);
@@ -220,7 +220,7 @@ bool BalanceSheet_SaveToFile(const char* outPath) {
} }
bool BalanceSheet_LoadFromFile(const char* inPath) { bool BalanceSheet_LoadFromFile(const char* inPath) {
if (!sheetMap) { return false; } if (!g_sheetMap) { return false; }
pthread_mutex_lock(&g_sheetLock); pthread_mutex_lock(&g_sheetLock);
char inFile[512]; char inFile[512];
@@ -247,15 +247,15 @@ bool BalanceSheet_LoadFromFile(const char* inPath) {
} }
void BalanceSheet_Print() { void BalanceSheet_Print() {
if (!sheetMap) { return; } if (!g_sheetMap) { return; }
pthread_mutex_lock(&g_sheetLock); pthread_mutex_lock(&g_sheetLock);
// Iterate through every entry // Iterate through every entry
khiter_t k; khiter_t k;
for (k = kh_begin(sheetMap); k != kh_end(sheetMap); ++k) { for (k = kh_begin(g_sheetMap); k != kh_end(g_sheetMap); ++k) {
if (kh_exist(sheetMap, k)) { if (kh_exist(g_sheetMap, k)) {
key32_t key = kh_key(sheetMap, k); key32_t key = kh_key(g_sheetMap, k);
balance_sheet_entry_t val = kh_val(sheetMap, k); balance_sheet_entry_t val = kh_val(g_sheetMap, k);
char balanceStr[80]; char balanceStr[80];
uint256_serialize(&val.balance, balanceStr); uint256_serialize(&val.balance, balanceStr);
@@ -273,8 +273,8 @@ void BalanceSheet_Print() {
} }
void BalanceSheet_Destroy() { void BalanceSheet_Destroy() {
kh_destroy(balance_sheet_map_m, sheetMap); kh_destroy(balance_sheet_map_m, g_sheetMap);
sheetMap = NULL; g_sheetMap = NULL;
pthread_mutex_destroy(&g_sheetLock); pthread_mutex_destroy(&g_sheetLock);
} }
src/block/block.c
+171 -75
View File
@@ -1,45 +1,130 @@
#include <block/block.h> #include <block/block.h>
#include <block/chain.h>
#include <autolykos2/autolykos2.h> #include <autolykos2/autolykos2.h>
#include <utils.h> #include <utils.h>
#include <stdlib.h> #include <stdlib.h>
#include <pthread.h>
/**
* The process-global mining DAG.
*
* Guarded by `g_powCtxLock` because generation frees and reallocates the buffer that hashing reads
* from: without the lock, an epoch rollover would pull the DAG out from under a miner mid-hash.
* Only the miner ever builds or reads this -- validation goes through the light path -- so the lock
* is essentially uncontended, and a node that does not mine never allocates a DAG at all.
**/
static Autolykos2Context* g_autolykos2Ctx = NULL; static Autolykos2Context* g_autolykos2Ctx = NULL;
static pthread_mutex_t g_powCtxLock = PTHREAD_MUTEX_INITIALIZER;
static uint64_t g_dagEpoch = 0;
// The seed the current DAG was generated from. Matching on epoch index and size is NOT enough: a
// reorg replaces the block an epoch's seed is derived from while leaving the epoch index and size
// unchanged, so a stale DAG would still look current and silently hash against the wrong lanes.
static uint8_t g_dagSeed[32];
static bool g_dagReady = false;
static Autolykos2Context* GetAutolykos2Ctx(void) { // Caller must hold `g_powCtxLock`.
static Autolykos2Context* GetAutolykos2CtxLocked(void) {
if (!g_autolykos2Ctx) { if (!g_autolykos2Ctx) {
g_autolykos2Ctx = Autolykos2_Create(); g_autolykos2Ctx = Autolykos2_Create();
if (!g_autolykos2Ctx) { if (!g_autolykos2Ctx) {
fprintf(stderr, "Failed to create Autolykos2 context\n"); fprintf(stderr, "Failed to create Autolykos2 context\n");
exit(1); exit(1);
} }
Autolykos2_DagAllocate(g_autolykos2Ctx, DAG_BASE_SIZE); // Deliberately no DagAllocate here. Allocating without generating leaves dag.len == 0, so
// every heavy hash fails -- which used to be indistinguishable from a valid proof, because
// the failure path handed back a zeroed hash that compares below every target.
} }
return g_autolykos2Ctx; return g_autolykos2Ctx;
} }
void Block_ShutdownPowContext(void) { void Block_ShutdownPowContext(void) {
pthread_mutex_lock(&g_powCtxLock);
if (g_autolykos2Ctx) { if (g_autolykos2Ctx) {
Autolykos2_Destroy(g_autolykos2Ctx); Autolykos2_Destroy(g_autolykos2Ctx);
g_autolykos2Ctx = NULL; g_autolykos2Ctx = NULL;
} }
g_dagReady = false;
pthread_mutex_unlock(&g_powCtxLock);
} }
bool Block_RebuildAutolykos2Dag(size_t dagBytes, const uint8_t seed32[32]) { bool Block_EnsureAutolykos2Dag(uint64_t epochIndex, size_t dagBytes, const uint8_t seed32[32]) {
if (!seed32 || dagBytes == 0) { if (!seed32 || dagBytes < 32u || (dagBytes % 32u) != 0u) {
return false; return false;
} }
Autolykos2Context* ctx = GetAutolykos2Ctx(); pthread_mutex_lock(&g_powCtxLock);
if (!ctx) {
return false; // Already built from exactly this seed at this size: generation is seconds of work, never redo
// it. The seed has to be part of the test -- see g_dagSeed.
if (g_dagReady && g_autolykos2Ctx && g_dagEpoch == epochIndex &&
Autolykos2_DagSize(g_autolykos2Ctx) == dagBytes &&
memcmp(g_dagSeed, seed32, 32) == 0) {
pthread_mutex_unlock(&g_powCtxLock);
return true;
} }
Autolykos2Context* ctx = GetAutolykos2CtxLocked();
g_dagReady = false; // the buffer is about to be invalid; no heavy hash may run against it
// Generation is one Blake2b per 64 bytes, single-threaded, so a multi-GiB DAG is tens of
// seconds. Say so rather than leaving the miner looking hung.
printf("Generating the epoch %llu mining DAG (%zu MiB), this takes a moment...\n",
(unsigned long long)epochIndex, dagBytes >> 20);
fflush(stdout);
Autolykos2_DagClear(ctx); Autolykos2_DagClear(ctx);
if (!Autolykos2_DagAllocate(ctx, dagBytes)) { const bool ok = Autolykos2_DagAllocate(ctx, dagBytes) && Autolykos2_DagGenerate(ctx, seed32);
if (ok) {
g_dagEpoch = epochIndex;
memcpy(g_dagSeed, seed32, 32);
g_dagReady = true;
}
pthread_mutex_unlock(&g_powCtxLock);
return ok;
}
bool Block_PowHashHeavy(const block_t* block, uint64_t epochIndex, size_t dagBytes,
const uint8_t seed32[32], uint8_t outHash[32]) {
if (!block || !seed32 || !outHash) {
return false; return false;
} }
return Autolykos2_DagGenerate(ctx, seed32); pthread_mutex_lock(&g_powCtxLock);
// Verifying the SEED here, not just the epoch and size, is what makes this impossible to
// misuse. A reorg changes the block an epoch's seed is derived from while the epoch index and
// size stay put, so an epoch+size check alone happily accepts a DAG built from the pre-reorg
// seed and returns a hash for the wrong lanes -- which shows up as a valid block failing PoW
// while a branch is being applied. A mismatch yields false and the caller derives the lanes
// from the seed instead.
const bool usable = g_dagReady && g_autolykos2Ctx && g_dagEpoch == epochIndex &&
Autolykos2_DagSize(g_autolykos2Ctx) == dagBytes &&
memcmp(g_dagSeed, seed32, 32) == 0;
const bool ok = usable &&
Autolykos2_Hash(
g_autolykos2Ctx,
(const uint8_t*)&block->header,
sizeof(block_header_t),
block->header.nonce,
block->header.blockNumber, // full 64-bit width; the light path takes uint64
outHash);
pthread_mutex_unlock(&g_powCtxLock);
return ok;
}
bool Block_PowHashLight(const block_t* block, size_t dagBytes, const uint8_t seed32[32], uint8_t outHash[32]) {
if (!block || !seed32 || !outHash) {
return false;
}
return Autolykos2_LightHashAtHeight(
seed32,
(const uint8_t*)&block->header,
sizeof(block_header_t),
block->header.nonce,
block->header.blockNumber,
dagBytes,
outHash);
} }
block_t* Block_Create() { block_t* Block_Create() {
@@ -48,7 +133,7 @@ block_t* Block_Create() {
return NULL; return NULL;
} }
memset(&block->header, 0, sizeof(block_header_t)); memset(&block->header, 0, sizeof(block_header_t));
block->transactions = DYNARR_CREATE(signed_transaction_t, 1); block->transactions = vector_create(sizeof(signed_transaction_t));
if (!block->transactions) { if (!block->transactions) {
free(block); free(block);
return NULL; return NULL;
@@ -75,13 +160,13 @@ void Block_CalculateMerkleRoot(const block_t* block, uint8_t* outHash) {
return; return;
} }
const size_t txCount = DynArr_size(block->transactions); const size_t txCount = vector_size(block->transactions);
if (txCount == 0) { if (txCount == 0) {
memset(outHash, 0, 32); memset(outHash, 0, 32);
return; return;
} }
if (txCount == 1) { if (txCount == 1) {
signed_transaction_t* tx = (signed_transaction_t*)DynArr_at(block->transactions, 0); signed_transaction_t* tx = (signed_transaction_t*)vector_get(block->transactions, 0);
Transaction_CalculateHash(tx, outHash); Transaction_CalculateHash(tx, outHash);
return; return;
} }
@@ -96,7 +181,7 @@ void Block_CalculateMerkleRoot(const block_t* block, uint8_t* outHash) {
} }
for (size_t i = 0; i < txCount; ++i) { for (size_t i = 0; i < txCount; ++i) {
signed_transaction_t* tx = (signed_transaction_t*)DynArr_at(block->transactions, i); signed_transaction_t* tx = (signed_transaction_t*)vector_get(block->transactions, i);
if (!tx) { if (!tx) {
free(current); free(current);
free(next); free(next);
@@ -133,36 +218,12 @@ void Block_CalculateMerkleRoot(const block_t* block, uint8_t* outHash) {
free(next); free(next);
} }
void Block_CalculateAutolykos2Hash(const block_t* block, uint8_t* outHash) {
if (!block || !outHash) {
return;
}
// PoW hash is computed from the block header, while canonical block hash remains SHA256.
Autolykos2Context* ctx = GetAutolykos2Ctx();
if (!ctx) {
memset(outHash, 0, 32);
return;
}
if (!Autolykos2_Hash(
ctx,
(const uint8_t*)&block->header,
sizeof(block_header_t),
block->header.nonce,
(uint32_t)block->header.blockNumber,
outHash
)) {
memset(outHash, 0, 32);
}
}
void Block_AddTransaction(block_t* block, signed_transaction_t* tx) { void Block_AddTransaction(block_t* block, signed_transaction_t* tx) {
if (!block || !tx || !block->transactions) { if (!block || !tx || !block->transactions) {
return; return;
} }
DynArr_push_back(block->transactions, tx); vector_push_back(block->transactions, tx);
} }
void Block_RemoveTransaction(block_t* block, uint8_t* txHash) { void Block_RemoveTransaction(block_t* block, uint8_t* txHash) {
@@ -170,12 +231,12 @@ void Block_RemoveTransaction(block_t* block, uint8_t* txHash) {
return; return;
} }
for (size_t i = 0; i < DynArr_size(block->transactions); i++) { for (size_t i = 0; i < vector_size(block->transactions); i++) {
signed_transaction_t* currentTx = (signed_transaction_t*)DynArr_at(block->transactions, i); signed_transaction_t* currentTx = (signed_transaction_t*)vector_get(block->transactions, i);
uint8_t currentTxHash[32]; uint8_t currentTxHash[32];
Transaction_CalculateHash(currentTx, currentTxHash); Transaction_CalculateHash(currentTx, currentTxHash);
if (memcmp(currentTxHash, txHash, 32) == 0) { if (memcmp(currentTxHash, txHash, 32) == 0) {
DynArr_remove(block->transactions, i); vector_pop_at(block->transactions, i);
return; return;
} }
} }
@@ -189,7 +250,8 @@ static int Uint256_CompareBE(const uint8_t a[32], const uint8_t b[32]) {
return 0; return 0;
} }
bool Block_HasValidProofOfWork(const block_t* block) { bool Block_HasValidProofOfWorkWithParams(const block_t* block, uint64_t epochIndex,
size_t dagBytes, const uint8_t seed32[32]) {
if (!block) { if (!block) {
return false; return false;
} }
@@ -199,12 +261,49 @@ bool Block_HasValidProofOfWork(const block_t* block) {
return false; return false;
} }
// Prefer the prebuilt DAG when it is provably the one for this block's epoch and size -- the
// miner keeps it warm, and reading a lane beats recomputing it -- otherwise derive the lanes
// from the epoch seed. The two produce identical hashes, so which one runs is invisible to
// consensus; only speed differs.
uint8_t hash[32]; uint8_t hash[32];
Block_CalculateAutolykos2Hash(block, hash); if (!Block_PowHashHeavy(block, epochIndex, dagBytes, seed32, hash) &&
!Block_PowHashLight(block, dagBytes, seed32, hash)) {
// Fail CLOSED. This used to hand back a zeroed hash on any failure and compare that to the
// target -- and zero is below every target, so a DAG that was missing, mis-sized or failed
// to build made the PoW check pass for every block instead of rejecting them.
return false;
}
return Uint256_CompareBE(hash, target) <= 0; return Uint256_CompareBE(hash, target) <= 0;
} }
bool Block_HasValidProofOfWork(const block_t* block, blockchain_t* chain) {
if (!block || !chain) {
return false;
}
size_t dagBytes = 0;
uint8_t seed[32];
if (!Chain_DagParamsForHeight(chain, block->header.blockNumber, &dagBytes, seed)) {
return false;
}
const uint64_t epochIndex = block->header.blockNumber / (uint64_t)EPOCH_LENGTH;
return Block_HasValidProofOfWorkWithParams(block, epochIndex, dagBytes, seed);
}
bool Block_HasValidVote(const block_t* block) {
if (!block) {
return false;
}
// Unrecognised vote values and non-zero spare bytes are rejected rather than ignored, so the
// header has no bits whose meaning is undefined and nothing to grind for extra nonce space.
return block->header.reserved[0] <= (uint8_t)DAG_VOTE_MAX &&
block->header.reserved[1] == 0u &&
block->header.reserved[2] == 0u;
}
bool Block_AllTransactionsValid(const block_t* block) { bool Block_AllTransactionsValid(const block_t* block) {
if (!block || !block->transactions) { if (!block || !block->transactions) {
return false; return false;
@@ -212,8 +311,8 @@ bool Block_AllTransactionsValid(const block_t* block) {
bool hasCoinbase = false; bool hasCoinbase = false;
for (size_t i = 0; i < DynArr_size(block->transactions); i++) { for (size_t i = 0; i < vector_size(block->transactions); i++) {
signed_transaction_t* tx = (signed_transaction_t*)DynArr_at(block->transactions, i); signed_transaction_t* tx = (signed_transaction_t*)vector_get(block->transactions, i);
if (!Transaction_Verify(tx)) { if (!Transaction_Verify(tx)) {
return false; return false;
} }
@@ -227,7 +326,7 @@ bool Block_AllTransactionsValid(const block_t* block) {
} }
} }
return true && hasCoinbase && DynArr_size(block->transactions) > 0; // Every block must have at least one transaction (the coinbase) return true && hasCoinbase && vector_size(block->transactions) > 0; // Every block must have at least one transaction (the coinbase)
} }
bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinbaseAmount, uint64_t* outTotalFees) { bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinbaseAmount, uint64_t* outTotalFees) {
@@ -239,8 +338,8 @@ bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinba
uint64_t totalFees = 0; uint64_t totalFees = 0;
uint8_t zeroAddress[32] = {0}; uint8_t zeroAddress[32] = {0};
for (size_t i = 0; i < DynArr_size(block->transactions); ++i) { for (size_t i = 0; i < vector_size(block->transactions); ++i) {
signed_transaction_t* tx = (signed_transaction_t*)DynArr_at(block->transactions, i); signed_transaction_t* tx = (signed_transaction_t*)vector_get(block->transactions, i);
if (!tx) { if (!tx) {
return false; return false;
} }
@@ -296,20 +395,29 @@ bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinba
return true; return true;
} }
bool Block_IsFullyValid(const block_t* block) { bool Block_HasValidStructure(const block_t* block) {
bool merkleValid = false; if (!block || !block->transactions) {
uint8_t calculatedMerkleRoot[32]; return false;
if (block && block->transactions) {
Block_CalculateMerkleRoot(block, calculatedMerkleRoot);
merkleValid = (memcmp(calculatedMerkleRoot, block->header.merkleRoot, 32) == 0);
} }
return Block_HasValidProofOfWork(block) && Block_AllTransactionsValid(block) && DynArr_size(block->transactions) > 0 && merkleValid; uint8_t calculatedMerkleRoot[32];
Block_CalculateMerkleRoot(block, calculatedMerkleRoot);
if (memcmp(calculatedMerkleRoot, block->header.merkleRoot, 32) != 0) {
return false;
}
return Block_HasValidVote(block) &&
Block_AllTransactionsValid(block) &&
vector_size(block->transactions) > 0;
}
bool Block_IsFullyValid(const block_t* block, blockchain_t* chain) {
return Block_HasValidStructure(block) && Block_HasValidProofOfWork(block, chain);
} }
void Block_Destroy(block_t* block) { void Block_Destroy(block_t* block) {
if (!block) return; if (!block) return;
DynArr_destroy(block->transactions); vector_destroy(&block->transactions);
free(block); free(block);
} }
@@ -332,9 +440,9 @@ void Block_Print(const block_t* block) {
} }
printf("\n"); printf("\n");
if (block->transactions) { if (block->transactions) {
printf("Transactions (%zu):\n", DynArr_size(block->transactions)); printf("Transactions (%zu):\n", vector_size(block->transactions));
for (size_t i = 0; i < DynArr_size(block->transactions); i++) { for (size_t i = 0; i < vector_size(block->transactions); i++) {
signed_transaction_t* tx = (signed_transaction_t*)DynArr_at(block->transactions, i); signed_transaction_t* tx = (signed_transaction_t*)vector_get(block->transactions, i);
if (tx) { if (tx) {
printf(" Tx #%zu: 1: %llu -> %02x%02x...%02x%02x, fee %llu\n 2: %llu -> %02x%02x...%02x%02x, fee %llu\n", printf(" Tx #%zu: 1: %llu -> %02x%02x...%02x%02x, fee %llu\n 2: %llu -> %02x%02x...%02x%02x, fee %llu\n",
i, i,
@@ -362,7 +470,7 @@ void Block_ShortPrint(const block_t* block) {
block->header.version, block->header.version,
block->header.prevHash[0], block->header.prevHash[1], block->header.prevHash[30], block->header.prevHash[31], block->header.prevHash[0], block->header.prevHash[1], block->header.prevHash[30], block->header.prevHash[31],
block->header.merkleRoot[0], block->header.merkleRoot[1], block->header.merkleRoot[30], block->header.merkleRoot[31], block->header.merkleRoot[0], block->header.merkleRoot[1], block->header.merkleRoot[30], block->header.merkleRoot[31],
block->transactions ? DynArr_size(block->transactions) : 0); vector_size(block->transactions));
} }
block_t* Block_Copy(const block_t* src) { block_t* Block_Copy(const block_t* src) {
@@ -371,25 +479,13 @@ block_t* Block_Copy(const block_t* src) {
if (!dst) return NULL; if (!dst) return NULL;
dst->header = src->header; dst->header = src->header;
if (src->transactions) { if (src->transactions) {
size_t txCount = DynArr_size(src->transactions); // Transactions own no memory of their own, so the vector has no destructor and a byte-wise
dst->transactions = DYNARR_CREATE(signed_transaction_t, txCount == 0 ? 1 : txCount); // deep copy is exactly an independent copy of the list.
dst->transactions = vector_deep_copy(src->transactions);
if (!dst->transactions) { if (!dst->transactions) {
free(dst); free(dst);
return NULL; return NULL;
} }
for (size_t i = 0; i < txCount; ++i) {
signed_transaction_t* tx = (signed_transaction_t*)DynArr_at(src->transactions, i);
if (!tx) {
DynArr_destroy(dst->transactions);
free(dst);
return NULL;
}
if (!DynArr_push_back(dst->transactions, tx)) {
DynArr_destroy(dst->transactions);
free(dst);
return NULL;
}
}
} else { } else {
dst->transactions = NULL; dst->transactions = NULL;
} }
src/block/chain.c
+1217 -167
View File
File diff suppressed because it is too large Load Diff
src/dynarr.c
-179
View File
@@ -1,179 +0,0 @@
#include <dynarr.h>
DynArr* DynArr_create(size_t elemSize, size_t capacity) {
DynArr* p = (DynArr*)malloc(sizeof(DynArr));
if (!p) return NULL;
p->elemSize = elemSize;
p->capacity = capacity;
p->data = malloc(elemSize * capacity);
if (!p->data) {
free(p);
return NULL;
}
p->size = 0;
return p;
}
// Reserve n blocks in arary; New size will be n, NOT size + n; Reserving less memory that current will fail, use prune instead.
void DynArr_reserve(DynArr* p, size_t n) {
if (n <= p->capacity) {
printf("reserve ignored; attempted to reserve less or equal to current capacity\n");
return;
}
if (n > DYNARR_MAX_CAPACITY) {
printf("reserve ignored; attempted to reserve over 32 bits\n");
return;
}
void* new_data = realloc(p->data, n * p->elemSize);
if (!new_data) {
printf("reserve failed\n");
exit(1);
}
p->data = new_data;
p->capacity = n;
}
// Push data into a new block at the end of the array; If value is NULL, the new block will be zeroed.
void* DynArr_push_back(DynArr* p, void* value) {
//if (value == NULL) {
// printf("push_back ignored; value is null");
// return NULL;
//}
if (p->size >= p->capacity) {
size_t new_cap = (p->capacity == 0) ? 1 : p->capacity * 2;
if (new_cap < p->capacity || new_cap > DYNARR_MAX_CAPACITY) {
printf("push_back ignored; capacity overflow\n");
return NULL;
}
void* new_data = realloc(p->data, new_cap * p->elemSize);
if (!new_data) {
printf("push failed\n");
exit(1);
}
p->capacity = new_cap;
p->data = new_data;
}
void* dst = (void*)((char*)p->data + (p->size * p->elemSize));
if (value == NULL) {
memset(dst, 0, p->elemSize); // Handle NULL value.
} else {
memcpy((char*)dst, value, p->elemSize);
}
p->size++;
return dst;
}
// Remove the last block in the array.
void DynArr_pop_back(DynArr* p) {
if (p->size == 0) {
printf("pop_back ignored; size is 0\n");
return;
}
p->size--; // Will automatically overwrite that memory naturally
}
// Remove first block from array.
void DynArr_pop_front(DynArr* p) {
if (p->size == 0) {
printf("pop_front ignored; size is 0\n");
return;
}
memmove(
(char*)p->data,
(char*)p->data + p->elemSize,
(p->size - 1) * p->elemSize
);
p->size--;
}
// Remove index from array. This moves all blocks after the index block.
void DynArr_remove(DynArr* p, size_t index) {
if (index >= p->size) return;
memmove(
(char*)p->data + (index * p->elemSize),
(char*)p->data + (index + 1) * p->elemSize,
(p->size - index - 1) * p->elemSize
);
p->size--;
}
// Erase the array. This will not free unused blocks.
void DynArr_erase(DynArr* p) {
p->size = 0;
}
// Prune and free unused blocks. If pruning to zero, ensure to reserve after.
void DynArr_prune(DynArr* p) {
void* new_data = realloc(p->data, (p->size == 0 ? 1 : p->size) * p->elemSize);
if (!new_data) {
printf("pruning failed\n");
exit(1);
}
p->data = new_data;
p->capacity = p->size;
}
// Get a pointer to a block by index
void* DynArr_at(DynArr* p, size_t index) {
if (index >= p->size) return NULL;
return (char*)p->data + (index * p->elemSize);
}
// Get the index by block pointer
size_t DynArr_at_ptr(DynArr* p, void* ptr) {
if (!p || !ptr) {
printf("invalid pointer\n");
exit(1);
}
for (size_t i = 0; i < p->size; i++) {
if ((void*)(((char*)p->data) + (i * p->elemSize)) == ptr) {
return i;
}
}
// If for some reason the array has 2^64 elements in it, then fuck it, I guess we'll just crash, I don't care.
return -1;
}
// Get size
size_t DynArr_size(DynArr* p) {
return p->size;
}
// Get element size
size_t DynArr_elemSize(DynArr* p) {
return p->elemSize;
}
// Get capacity
size_t DynArr_capacity(DynArr* p) {
return p->capacity;
}
void DynArr_destroy(DynArr* p) {
if (!p) return;
free(p->data);
free(p);
}
void* DynArr_c_arr(DynArr* p) {
return p->data;
}
src/dynset.c
-58
View File
@@ -1,58 +0,0 @@
#include <dynset.h>
DynSet* DynSet_Create(size_t elemSize) {
DynSet* set = (DynSet*)malloc(sizeof(DynSet));
if (!set) {
return NULL;
}
set->arr = DynArr_create(elemSize, 1);
if (!set->arr) {
free(set);
return NULL;
}
return set;
}
void DynSet_Destroy(DynSet* set) {
if (set) {
DynArr_destroy(set->arr);
free(set);
}
}
int DynSet_Insert(DynSet* set, const void* element) {
if (DynSet_Contains(set, element)) {
return 0; // Element already exists
}
return DynArr_push_back(set->arr, element) != NULL;
}
int DynSet_Contains(DynSet* set, const void* element) {
size_t size = DynArr_size(set->arr);
for (size_t i = 0; i < size; i++) {
void* current = DynArr_at(set->arr, i);
if (memcmp(current, element, set->arr->elemSize) == 0) {
return 1; // Found
}
}
return 0; // Not found
}
size_t DynSet_Size(DynSet* set) {
return DynArr_size(set->arr);
}
void* DynSet_Get(DynSet* set, size_t index) {
return DynArr_at(set->arr, index);
}
void DynSet_Remove(DynSet* set, const void* element) {
size_t size = DynArr_size(set->arr);
for (size_t i = 0; i < size; i++) {
void* current = DynArr_at(set->arr, i);
if (memcmp(current, element, set->arr->elemSize) == 0) {
DynArr_remove(set->arr, i);
return;
}
}
}
src/main.c
+674 -280
View File
File diff suppressed because it is too large Load Diff
src/nets/fetch_scheduler.c
+41 -12
View File
@@ -1,24 +1,53 @@
#include <nets/fetch_scheduler.h> #include <nets/fetch_scheduler.h>
#include <constants.h> #include <constants.h>
#include <math.h>
// Note: floating point is used intentionally here for readability and // Integer-only on purpose. This penalty gates fork choice (see Chain_ReplaceBranch), so every node
// because the final penalty is rounded to whole blocks. This keeps the // must compute the exact same number of blocks from the same reorg depth. The previous
// implementation straightforward while avoiding subtle integer overflow // implementation used double/pow/ceil, which is not reproducible across platforms and compilers.
// for large exponents. If desired, replace with fixed-point arithmetic.
uint64_t FetchScheduler_ComputeReorgPenaltyBlocks(uint64_t delayBlocks) { uint64_t FetchScheduler_ComputeReorgPenaltyBlocks(uint64_t delayBlocks) {
if (delayBlocks <= REORG_PENALTY_GRACE_BLOCKS) { if (delayBlocks <= REORG_PENALTY_GRACE_BLOCKS) {
return 0ULL; return 0ULL;
} }
double B = (double)delayBlocks; uint64_t depth = delayBlocks;
double factor = REORG_PENALTY_FACTOR; if (depth > REORG_PENALTY_MAX_DEPTH) {
double exp = REORG_PENALTY_EXPONENT; depth = REORG_PENALTY_MAX_DEPTH;
double timeScale = ((double)TARGET_BLOCK_TIME) / REORG_PENALTY_REF_BLOCK_TIME; }
double raw = factor * pow(B, exp) * timeScale; // depth^EXPONENT, saturating rather than wrapping.
if (raw < 0.0) raw = 0.0; uint64_t raised = 1ULL;
for (uint32_t i = 0; i < REORG_PENALTY_EXPONENT; ++i) {
if (depth != 0ULL && raised > UINT64_MAX / depth) {
return UINT64_MAX;
}
raised *= depth;
}
// Scale by theta and by the block-time ratio, as one fraction so there is a single rounding
// step: penalty = ceil(raised * FACTOR_NUM * REF_BLOCK_TIME / (FACTOR_DEN * TARGET_BLOCK_TIME))
//
// REF_BLOCK_TIME is the NUMERATOR and TARGET_BLOCK_TIME the DENOMINATOR, not the other way
// round. The result is a count of BLOCKS, so the wall-clock protection it buys is
// penalty(d) * TARGET_BLOCK_TIME ~= d^p * REF_BLOCK_TIME -- TARGET_BLOCK_TIME cancels, and the
// protection is the same number of seconds whatever the block time is. Inverting these two
// makes wall-clock protection scale as TARGET_BLOCK_TIME^2, so shortening the block time
// silently weakens reorg protection. Do not "simplify" this back.
const uint64_t numeratorScale = REORG_PENALTY_FACTOR_NUM * REORG_PENALTY_REF_BLOCK_TIME;
const uint64_t denominator = REORG_PENALTY_FACTOR_DEN * (uint64_t)TARGET_BLOCK_TIME;
if (denominator == 0ULL) {
return 0ULL;
}
if (numeratorScale != 0ULL && raised > UINT64_MAX / numeratorScale) {
return UINT64_MAX;
}
const uint64_t numerator = raised * numeratorScale;
// Ceiling division without overflowing on the +denominator-1 term.
uint64_t penalty = numerator / denominator;
if (numerator % denominator != 0ULL) {
penalty++;
}
uint64_t penalty = (uint64_t)ceil(raw);
return penalty; return penalty;
} }
src/nets/net_node.c
+895 -107
View File
File diff suppressed because it is too large Load Diff
src/nets/nodediscovery.c
+753
View File
@@ -0,0 +1,753 @@
#include <nets/nodediscovery.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <pthread.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <ifaddrs.h>
#include <constants.h>
#include <dlibc/vector.h>
#include <numgen.h>
#include <runtime_state.h>
#include <utils.h>
// Wire layout of a single peer endpoint inside a PEERS payload:
// [uint8 family (4 or 6)][uint8 ip[16]][uint16 port (host order)] -> 19 bytes
// (Host-endian raw layout, consistent with the rest of the protocol.)
#define DISCOVERY_WIRE_ENTRY_SIZE (1 + 16 + 2)
typedef enum {
DISCOVERY_STATE_NEW = 0, // learned, not yet pinged
DISCOVERY_STATE_PINGED, // first ping in flight, reachability unknown
DISCOVERY_STATE_REACHABLE, // pong received, latency known
DISCOVERY_STATE_CONNECTED, // currently a live outbound connection
DISCOVERY_STATE_UNREACHABLE // ping timed out
} discovery_state_t;
typedef struct {
struct sockaddr_storage addr; // listen endpoint (port already set to the peer's listen port)
uint64_t pingMs; // measured UDP RTT, UINT64_MAX if unknown
uint64_t nodeId; // identity of the node behind this endpoint, 0 while unknown
uint32_t hop; // distance from us (0 = directly connected)
discovery_state_t state;
int pingPending; // 1 while a ping is outstanding (matched by address on pong/timeout).
// The UDP layer generates its own nonce, so we can't match by nonce here.
uint64_t lastPingMs; // when we last sent a ping
uint64_t lastQueryMs; // when we last sent GET_PEERS to it
} discovered_peer_t;
// When we last dialed an endpoint. Kept outside the peer table on purpose: a peer entry is struck
// the moment its connection drops, and if the dial history went with it, an endpoint that hangs up
// on us would be re-learned through gossip and redialed on every single tick.
typedef struct {
struct sockaddr_storage addr;
uint64_t lastMs;
} discovery_attempt_t;
struct node_discovery {
net_node_t* node;
udp_node_t* udpNode;
vector_t* peers; // of discovered_peer_t
vector_t* selfEndpoints; // of struct sockaddr_storage - our own listen endpoints
vector_t* connectAttempts; // of discovery_attempt_t
pthread_mutex_t lock;
};
// ---- small helpers (most assume the caller holds disc->lock) ------------------------------
static int Discovery_AddrEqual(const struct sockaddr_storage* a, const struct sockaddr_storage* b) {
if (a->ss_family != b->ss_family) return 0;
if (a->ss_family == AF_INET) {
const struct sockaddr_in* x = (const struct sockaddr_in*)a;
const struct sockaddr_in* y = (const struct sockaddr_in*)b;
return x->sin_port == y->sin_port &&
memcmp(&x->sin_addr, &y->sin_addr, sizeof(struct in_addr)) == 0;
}
if (a->ss_family == AF_INET6) {
const struct sockaddr_in6* x = (const struct sockaddr_in6*)a;
const struct sockaddr_in6* y = (const struct sockaddr_in6*)b;
return x->sin6_port == y->sin6_port &&
memcmp(&x->sin6_addr, &y->sin6_addr, sizeof(struct in6_addr)) == 0;
}
return 0;
}
// Rejects endpoints that can never be dialed as written. IPv6 in particular hands us plenty of
// these: link-local addresses are meaningless without the scope id (which the wire format does not
// carry), and the unspecified/multicast ranges are never a peer. Loopback stays allowed so several
// nodes can still be run on one machine on different ports.
static int Discovery_IsUsableAddr(const struct sockaddr_storage* addr) {
if (addr->ss_family == AF_INET) {
const struct sockaddr_in* a = (const struct sockaddr_in*)addr;
if (a->sin_port == 0) return 0;
uint32_t host = ntohl(a->sin_addr.s_addr);
if (host == INADDR_ANY || host == INADDR_BROADCAST) return 0;
if ((host >> 28) == 0xE) return 0; // 224.0.0.0/4 multicast
if ((host & 0xFFFF0000u) == 0xA9FE0000u) return 0; // 169.254.0.0/16 link-local
return 1;
}
if (addr->ss_family == AF_INET6) {
const struct sockaddr_in6* a = (const struct sockaddr_in6*)addr;
if (a->sin6_port == 0) return 0;
if (IN6_IS_ADDR_UNSPECIFIED(&a->sin6_addr)) return 0;
if (IN6_IS_ADDR_MULTICAST(&a->sin6_addr)) return 0;
if (IN6_IS_ADDR_LINKLOCAL(&a->sin6_addr)) return 0; // unusable without a scope id
if (IN6_IS_ADDR_SITELOCAL(&a->sin6_addr)) return 0; // deprecated fec0::/10
return 1;
}
return 0;
}
// Rewrites an IPv4-mapped IPv6 endpoint (::ffff:a.b.c.d) as plain IPv4, so the same host never
// occupies two entries. Matches the normalisation Node_ConnListenEndpoint does.
static void Discovery_NormaliseAddr(struct sockaddr_storage* addr) {
if (addr->ss_family != AF_INET6) return;
struct sockaddr_in6* a = (struct sockaddr_in6*)addr;
if (!IN6_IS_ADDR_V4MAPPED(&a->sin6_addr)) return;
struct in_addr v4;
memcpy(&v4, ((const uint8_t*)&a->sin6_addr) + 12, sizeof(v4));
uint16_t port = a->sin6_port;
memset(addr, 0, sizeof(*addr));
struct sockaddr_in* o = (struct sockaddr_in*)addr;
o->sin_family = AF_INET;
o->sin_addr = v4;
o->sin_port = port;
}
// Returns non-zero if addr is one of our own listen endpoints. Caller holds disc->lock.
static int Discovery_IsSelfUnlocked(node_discovery_t* disc, const struct sockaddr_storage* addr) {
size_t n = vector_size(disc->selfEndpoints);
for (size_t i = 0; i < n; ++i) {
const struct sockaddr_storage* self = (const struct sockaddr_storage*)vector_get_const(disc->selfEndpoints, i);
if (self && Discovery_AddrEqual(self, addr)) return 1;
}
return 0;
}
// Adds addr to the self set if not already there. Caller holds disc->lock.
static void Discovery_AddSelfUnlocked(node_discovery_t* disc, const struct sockaddr_storage* addr) {
if (Discovery_IsSelfUnlocked(disc, addr)) return;
vector_push_back(disc->selfEndpoints, addr);
}
// Seeds the self set with (local interface address, our listen port) for every address this host
// carries. A multi-homed host - the normal case under IPv6, where a machine holds a global, a
// temporary privacy and a link-local address at once - is otherwise unable to tell its own
// endpoints from a peer's when they come back around through peer exchange.
static void Discovery_SeedSelfEndpoints(node_discovery_t* disc) {
struct ifaddrs* ifa = NULL;
if (getifaddrs(&ifa) != 0 || !ifa) return;
for (struct ifaddrs* it = ifa; it; it = it->ifa_next) {
if (!it->ifa_addr) continue;
struct sockaddr_storage ep;
memset(&ep, 0, sizeof(ep));
if (it->ifa_addr->sa_family == AF_INET) {
struct sockaddr_in* o = (struct sockaddr_in*)&ep;
memcpy(o, it->ifa_addr, sizeof(struct sockaddr_in));
o->sin_port = htons(g_listenPort);
} else if (it->ifa_addr->sa_family == AF_INET6) {
struct sockaddr_in6* o = (struct sockaddr_in6*)&ep;
memcpy(o, it->ifa_addr, sizeof(struct sockaddr_in6));
o->sin6_port = htons(g_listenPort);
o->sin6_scope_id = 0; // endpoints on the wire are scopeless; compare them the same way
} else {
continue;
}
Discovery_NormaliseAddr(&ep);
Discovery_AddSelfUnlocked(disc, &ep);
}
freeifaddrs(ifa);
}
static discovered_peer_t* Discovery_FindPtr(node_discovery_t* disc, const struct sockaddr_storage* addr) {
size_t n = vector_size(disc->peers);
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)vector_get(disc->peers, i);
if (p && Discovery_AddrEqual(&p->addr, addr)) return p;
}
return NULL;
}
// Insert addr if not already present. Returns a pointer to the (existing or new) entry, or NULL if
// the address is unusable, is one of our own, or the table is full. Note: the returned pointer is
// invalidated by any later push_back.
static discovered_peer_t* Discovery_Upsert(node_discovery_t* disc, const struct sockaddr_storage* addr, uint32_t hop) {
if (!Discovery_IsUsableAddr(addr)) return NULL;
if (Discovery_IsSelfUnlocked(disc, addr)) return NULL; // never track, ping or dial ourselves
discovered_peer_t* existing = Discovery_FindPtr(disc, addr);
if (existing) {
if (hop < existing->hop) existing->hop = hop; // keep the shortest known distance
return existing;
}
if (vector_size(disc->peers) >= DISCOVERY_MAX_KNOWN_PEERS) return NULL;
discovered_peer_t np;
memset(&np, 0, sizeof(np));
np.addr = *addr;
np.pingMs = UINT64_MAX;
np.nodeId = 0;
np.hop = hop;
np.state = DISCOVERY_STATE_NEW;
// Checked: on a failed push, back() would hand the caller the previous peer instead.
if (vector_push_back(disc->peers, &np) != 0) return NULL;
return (discovered_peer_t*)vector_back(disc->peers);
}
// Returns non-zero if addr may be dialed again, i.e. we have not tried it within the retry window.
// Caller holds disc->lock.
static int Discovery_ConnectCooledDown(node_discovery_t* disc, const struct sockaddr_storage* addr, uint64_t now) {
size_t n = vector_size(disc->connectAttempts);
for (size_t i = 0; i < n; ++i) {
const discovery_attempt_t* a = (const discovery_attempt_t*)vector_get_const(disc->connectAttempts, i);
if (a && Discovery_AddrEqual(&a->addr, addr)) {
return (now - a->lastMs) >= DISCOVERY_CONNECT_RETRY_MS;
}
}
return 1; // never dialed
}
// Stamps a dial attempt against addr, evicting the stalest record once the table is full.
// Caller holds disc->lock.
static void Discovery_NoteConnectAttempt(node_discovery_t* disc, const struct sockaddr_storage* addr, uint64_t now) {
size_t n = vector_size(disc->connectAttempts);
size_t oldestIdx = 0;
uint64_t oldestMs = UINT64_MAX;
for (size_t i = 0; i < n; ++i) {
discovery_attempt_t* a = (discovery_attempt_t*)vector_get(disc->connectAttempts, i);
if (!a) continue;
if (Discovery_AddrEqual(&a->addr, addr)) {
a->lastMs = now;
return;
}
if (a->lastMs < oldestMs) {
oldestMs = a->lastMs;
oldestIdx = i;
}
}
if (n >= DISCOVERY_MAX_KNOWN_PEERS) {
discovery_attempt_t* victim = (discovery_attempt_t*)vector_get(disc->connectAttempts, oldestIdx);
if (victim) {
victim->addr = *addr;
victim->lastMs = now;
}
return;
}
discovery_attempt_t na;
memset(&na, 0, sizeof(na));
na.addr = *addr;
na.lastMs = now;
vector_push_back(disc->connectAttempts, &na);
}
// Drops the entry for addr, if any. Caller holds disc->lock.
static void Discovery_RemoveUnlocked(node_discovery_t* disc, const struct sockaddr_storage* addr) {
size_t n = vector_size(disc->peers);
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)vector_get(disc->peers, i);
if (p && Discovery_AddrEqual(&p->addr, addr)) {
vector_pop_at(disc->peers, i);
return;
}
}
}
static int Discovery_AddrToWire(const struct sockaddr_storage* addr, unsigned char out[DISCOVERY_WIRE_ENTRY_SIZE]) {
memset(out, 0, DISCOVERY_WIRE_ENTRY_SIZE);
if (addr->ss_family == AF_INET) {
const struct sockaddr_in* a = (const struct sockaddr_in*)addr;
out[0] = 4;
memcpy(out + 1, &a->sin_addr, sizeof(struct in_addr));
uint16_t port = ntohs(a->sin_port);
memcpy(out + 1 + 16, &port, sizeof(port));
return 1;
}
if (addr->ss_family == AF_INET6) {
const struct sockaddr_in6* a = (const struct sockaddr_in6*)addr;
out[0] = 6;
memcpy(out + 1, &a->sin6_addr, sizeof(struct in6_addr));
uint16_t port = ntohs(a->sin6_port);
memcpy(out + 1 + 16, &port, sizeof(port));
return 1;
}
return 0;
}
static int Discovery_WireToAddr(const unsigned char in[DISCOVERY_WIRE_ENTRY_SIZE], struct sockaddr_storage* out) {
memset(out, 0, sizeof(*out));
uint8_t fam = in[0];
uint16_t port;
memcpy(&port, in + 1 + 16, sizeof(port));
if (fam == 4) {
struct sockaddr_in* a = (struct sockaddr_in*)out;
a->sin_family = AF_INET;
memcpy(&a->sin_addr, in + 1, sizeof(struct in_addr));
a->sin_port = htons(port);
return port != 0;
}
if (fam == 6) {
struct sockaddr_in6* a = (struct sockaddr_in6*)out;
a->sin6_family = AF_INET6;
memcpy(&a->sin6_addr, in + 1, sizeof(struct in6_addr));
a->sin6_port = htons(port);
Discovery_NormaliseAddr(out); // a v4-mapped sender must not become a second entry
return port != 0;
}
return 0;
}
static int Discovery_AddrToIpPort(const struct sockaddr_storage* addr, char* ipOut, size_t ipLen, unsigned short* portOut) {
if (addr->ss_family == AF_INET) {
const struct sockaddr_in* a = (const struct sockaddr_in*)addr;
if (!inet_ntop(AF_INET, &a->sin_addr, ipOut, (socklen_t)ipLen)) return 0;
*portOut = ntohs(a->sin_port);
return 1;
}
if (addr->ss_family == AF_INET6) {
const struct sockaddr_in6* a = (const struct sockaddr_in6*)addr;
if (!inet_ntop(AF_INET6, &a->sin6_addr, ipOut, (socklen_t)ipLen)) return 0;
*portOut = ntohs(a->sin6_port);
return 1;
}
return 0;
}
// ---- lifecycle ---------------------------------------------------------------------------
node_discovery_t* NodeDiscovery_Create(net_node_t* node, udp_node_t* udpNode) {
if (!node || !udpNode) return NULL;
node_discovery_t* disc = (node_discovery_t*)malloc(sizeof(node_discovery_t));
if (!disc) return NULL;
memset(disc, 0, sizeof(*disc));
disc->node = node;
disc->udpNode = udpNode;
disc->peers = vector_create(sizeof(discovered_peer_t));
if (!disc->peers) {
free(disc);
return NULL;
}
disc->selfEndpoints = vector_create(sizeof(struct sockaddr_storage));
if (!disc->selfEndpoints) {
vector_destroy(&disc->peers);
free(disc);
return NULL;
}
disc->connectAttempts = vector_create(sizeof(discovery_attempt_t));
if (!disc->connectAttempts) {
vector_destroy(&disc->selfEndpoints);
vector_destroy(&disc->peers);
free(disc);
return NULL;
}
pthread_mutex_init(&disc->lock, NULL);
// Nothing else is running yet, so the self set can be seeded without taking the lock.
Discovery_SeedSelfEndpoints(disc);
return disc;
}
void NodeDiscovery_Destroy(node_discovery_t* disc) {
if (!disc) return;
vector_destroy(&disc->peers);
vector_destroy(&disc->selfEndpoints);
vector_destroy(&disc->connectAttempts);
pthread_mutex_destroy(&disc->lock);
free(disc);
}
// ---- UDP latency callbacks ---------------------------------------------------------------
void NodeDiscovery_OnPong(node_discovery_t* disc, const struct sockaddr_storage* from, uint64_t nonce, uint64_t rttMs) {
if (!disc || !from) return;
(void)nonce; // UDP layer owns the nonce; we match the peer by its reply address instead.
pthread_mutex_lock(&disc->lock);
discovered_peer_t* p = Discovery_FindPtr(disc, from);
if (p && p->pingPending) {
p->pingMs = rttMs;
p->pingPending = 0;
if (p->state == DISCOVERY_STATE_PINGED) p->state = DISCOVERY_STATE_REACHABLE;
}
pthread_mutex_unlock(&disc->lock);
}
void NodeDiscovery_OnPingTimeout(node_discovery_t* disc, const struct sockaddr_storage* dest, uint64_t nonce) {
if (!disc || !dest) return;
(void)nonce; // matched by the destination address we pinged
pthread_mutex_lock(&disc->lock);
discovered_peer_t* p = Discovery_FindPtr(disc, dest);
if (p && p->pingPending) {
p->pingPending = 0;
// Only demote a peer whose reachability was still unknown; a refresh ping that times
// out on an already-connected/reachable peer must not drop it.
if (p->state == DISCOVERY_STATE_PINGED) p->state = DISCOVERY_STATE_UNREACHABLE;
}
pthread_mutex_unlock(&disc->lock);
}
// ---- TCP peer exchange -------------------------------------------------------------------
void NodeDiscovery_OnGetPeers(node_discovery_t* disc, tcp_connection_t* fromConn) {
if (!disc || !fromConn) return;
// Snapshot our current peers' listen endpoints (inbound + outbound) and their identities.
struct sockaddr_storage all[MAX_CONS * 2];
uint64_t allIds[MAX_CONS * 2];
size_t total = Node_GetPeerEndpoints(disc->node, all, allIds, sizeof(all) / sizeof(all[0]));
struct sockaddr_storage reqEndpoint;
int haveReq = Node_ConnListenEndpoint(fromConn, &reqEndpoint);
uint64_t reqNodeId = Node_ConnPeerNodeId(fromConn);
// Build the response payload: [uint16 count][entries...], capped and sampled for spread.
unsigned char payload[sizeof(uint16_t) + DISCOVERY_PEERS_RESPONSE_CAP * DISCOVERY_WIRE_ENTRY_SIZE];
size_t offset = sizeof(uint16_t);
uint16_t count = 0;
size_t startIdx = total ? (size_t)(random_four_byte() % total) : 0;
for (size_t k = 0; k < total && count < DISCOVERY_PEERS_RESPONSE_CAP; ++k) {
size_t idx = (startIdx + k) % total;
// Don't tell them about themselves. Matching on identity as well as on the endpoint they
// reached us from matters: a multi-homed peer is known to us under several addresses, and
// handing one of its own back to it is what makes it discover, ping and dial itself.
if (haveReq && Discovery_AddrEqual(&all[idx], &reqEndpoint)) continue;
if (reqNodeId != 0 && allIds[idx] == reqNodeId) continue;
unsigned char entry[DISCOVERY_WIRE_ENTRY_SIZE];
if (!Discovery_AddrToWire(&all[idx], entry)) continue;
memcpy(payload + offset, entry, DISCOVERY_WIRE_ENTRY_SIZE);
offset += DISCOVERY_WIRE_ENTRY_SIZE;
count++;
}
memcpy(payload, &count, sizeof(count));
Node_SendPacket(disc->node, fromConn, PACKET_TYPE_PEERS, payload, offset);
}
void NodeDiscovery_OnPeersReceived(node_discovery_t* disc, tcp_connection_t* fromConn, const unsigned char* payload, size_t payloadLen) {
if (!disc || !payload || payloadLen < sizeof(uint16_t)) return;
uint16_t count;
memcpy(&count, payload, sizeof(count));
size_t need = sizeof(uint16_t) + (size_t)count * DISCOVERY_WIRE_ENTRY_SIZE;
if (payloadLen < need) return; // malformed / truncated
// Determine the hop distance of the peer that answered, so its peers land one hop further out.
struct sockaddr_storage srcEndpoint;
int haveSrc = fromConn ? Node_ConnListenEndpoint(fromConn, &srcEndpoint) : 0;
pthread_mutex_lock(&disc->lock);
uint32_t srcHop = 0;
if (haveSrc) {
discovered_peer_t* srcp = Discovery_FindPtr(disc, &srcEndpoint);
if (srcp) srcHop = srcp->hop;
}
uint32_t newHop = srcHop + 1;
// Fold in at most DISCOVERY_FANOUT *new* endpoints (a couple per node -> keeps the crawl spread).
if (newHop <= DISCOVERY_MAX_HOPS) {
int added = 0;
for (uint16_t i = 0; i < count && added < DISCOVERY_FANOUT; ++i) {
const unsigned char* entry = payload + sizeof(uint16_t) + (size_t)i * DISCOVERY_WIRE_ENTRY_SIZE;
struct sockaddr_storage ep;
if (!Discovery_WireToAddr(entry, &ep)) continue;
if (Discovery_FindPtr(disc, &ep) != NULL) continue; // already known -> doesn't count toward fanout
if (Discovery_Upsert(disc, &ep, newHop) != NULL) added++;
}
}
pthread_mutex_unlock(&disc->lock);
}
void NodeDiscovery_RemovePeer(node_discovery_t* disc, const struct sockaddr_storage* endpoint) {
if (!disc || !endpoint) return;
pthread_mutex_lock(&disc->lock);
size_t n = vector_size(disc->peers);
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)vector_get(disc->peers, i);
if (p && Discovery_AddrEqual(&p->addr, endpoint)) {
char ip[INET6_ADDRSTRLEN] = {0};
unsigned short port = 0;
Discovery_AddrToIpPort(&p->addr, ip, sizeof(ip), &port);
printf("NodeDiscovery: struck disconnected peer %s:%u from peer list\n", ip, port);
vector_pop_at(disc->peers, i);
break;
}
}
pthread_mutex_unlock(&disc->lock);
}
void NodeDiscovery_NoteIdentity(node_discovery_t* disc, const struct sockaddr_storage* endpoint, uint64_t nodeId) {
if (!disc || !endpoint || nodeId == 0) return;
pthread_mutex_lock(&disc->lock);
if (nodeId == g_localNodeId) {
// The peer on the other end is us under one of our own addresses. Record it and drop it so
// discovery stops treating it as a peer.
Discovery_AddSelfUnlocked(disc, endpoint);
Discovery_RemoveUnlocked(disc, endpoint);
} else {
// Learn the endpoint if we did not already know it - a peer that dialled us is a perfectly
// good discovery candidate, and we now know both its listen endpoint and its identity.
discovered_peer_t* p = Discovery_Upsert(disc, endpoint, 0);
if (p) p->nodeId = nodeId;
}
pthread_mutex_unlock(&disc->lock);
}
void NodeDiscovery_MarkSelfEndpoint(node_discovery_t* disc, const struct sockaddr_storage* endpoint) {
if (!disc || !endpoint) return;
pthread_mutex_lock(&disc->lock);
Discovery_AddSelfUnlocked(disc, endpoint);
Discovery_RemoveUnlocked(disc, endpoint);
pthread_mutex_unlock(&disc->lock);
}
int NodeDiscovery_IsSelfEndpoint(node_discovery_t* disc, const struct sockaddr_storage* endpoint) {
if (!disc || !endpoint) return 0;
pthread_mutex_lock(&disc->lock);
int isSelf = Discovery_IsSelfUnlocked(disc, endpoint);
pthread_mutex_unlock(&disc->lock);
return isSelf;
}
// ---- periodic tick -----------------------------------------------------------------------
void NodeDiscovery_Iterate(node_discovery_t* disc) {
if (!disc || !disc->node || !disc->udpNode) return;
uint64_t now = get_current_time_ms();
// Snapshot current outbound connections and their listen endpoints (used for querying and
// for the "already connected?" checks below).
tcp_connection_t* outConns[MAX_CONS];
size_t outCount = 0;
Node_GetClientList(disc->node, outConns, &outCount);
struct sockaddr_storage outEndpoints[MAX_CONS];
uint64_t outNodeIds[MAX_CONS];
size_t outEpCount = 0;
for (size_t i = 0; i < outCount; ++i) {
struct sockaddr_storage ep;
if (Node_ConnListenEndpoint(outConns[i], &ep)) {
outNodeIds[outEpCount] = Node_ConnPeerNodeId(outConns[i]);
outEndpoints[outEpCount++] = ep;
}
}
// Deferred network actions, collected under the lock and executed after releasing it
// (Node_ConnectPeer / Node_SendPacket must not run while holding disc->lock).
tcp_connection_t* toQuery[MAX_CONS];
size_t toQueryCount = 0;
struct { char ip[INET6_ADDRSTRLEN]; unsigned short port; } toConnect[MAX_CONS];
size_t toConnectCount = 0;
pthread_mutex_lock(&disc->lock);
// 1. Seed: upsert connected (outbound) peers as CONNECTED at hop 0.
for (size_t i = 0; i < outEpCount; ++i) {
discovered_peer_t* p = Discovery_Upsert(disc, &outEndpoints[i], 0);
if (p) {
p->hop = 0;
p->state = DISCOVERY_STATE_CONNECTED;
if (outNodeIds[i] != 0) p->nodeId = outNodeIds[i];
}
}
// Demote entries still marked CONNECTED that are no longer in the outbound set.
{
size_t n = vector_size(disc->peers);
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)vector_get(disc->peers, i);
if (!p || p->state != DISCOVERY_STATE_CONNECTED) continue;
int stillConnected = 0;
for (size_t j = 0; j < outEpCount; ++j) {
if (Discovery_AddrEqual(&p->addr, &outEndpoints[j])) { stillConnected = 1; break; }
}
if (!stillConnected) {
p->state = (p->pingMs != UINT64_MAX) ? DISCOVERY_STATE_REACHABLE : DISCOVERY_STATE_NEW;
}
}
}
// 2. Ping NEW peers (and refresh stale REACHABLE ones), capped per tick.
{
int pings = 0;
size_t n = vector_size(disc->peers);
for (size_t i = 0; i < n && pings < DISCOVERY_MAX_PINGS_PER_TICK; ++i) {
discovered_peer_t* p = (discovered_peer_t*)vector_get(disc->peers, i);
if (!p) continue;
int shouldPing = 0;
if (!p->pingPending) {
if (p->state == DISCOVERY_STATE_NEW) {
shouldPing = 1;
} else if (p->state == DISCOVERY_STATE_REACHABLE &&
(now - p->lastPingMs) > DISCOVERY_PING_REFRESH_MS) {
shouldPing = 1;
}
}
if (!shouldPing) continue;
p->pingPending = 1;
p->lastPingMs = now;
if (p->state == DISCOVERY_STATE_NEW) p->state = DISCOVERY_STATE_PINGED;
UdpNode_SendPing(disc->udpNode, &p->addr);
pings++;
}
}
// 3. Timeout backstop (in case the UDP layer's own timeout callback is missed).
{
size_t n = vector_size(disc->peers);
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)vector_get(disc->peers, i);
if (!p) continue;
if (p->state == DISCOVERY_STATE_PINGED && p->pingPending &&
(now - p->lastPingMs) > DISCOVERY_PING_TIMEOUT_MS) {
p->pingPending = 0;
p->state = DISCOVERY_STATE_UNREACHABLE;
}
}
}
// 4. Query up to DISCOVERY_FANOUT connected peers with GET_PEERS, preferring the lowest ping
// and skipping ones we queried recently or that are already at the hop horizon.
{
int queries = 0;
while (queries < DISCOVERY_FANOUT) {
size_t bestIdx = outCount; // sentinel = none
uint64_t bestPing = UINT64_MAX;
for (size_t i = 0; i < outCount; ++i) {
struct sockaddr_storage ep;
if (!Node_ConnListenEndpoint(outConns[i], &ep)) continue;
discovered_peer_t* p = Discovery_FindPtr(disc, &ep);
if (!p) continue;
if (p->hop >= DISCOVERY_MAX_HOPS) continue;
if (p->lastQueryMs != 0 && (now - p->lastQueryMs) < DISCOVERY_QUERY_INTERVAL_MS) continue;
if (bestIdx == outCount || p->pingMs < bestPing) {
bestIdx = i;
bestPing = p->pingMs;
}
}
if (bestIdx == outCount) break; // nothing eligible
// Mark queried so it isn't picked again this tick, and queue the send.
struct sockaddr_storage ep;
if (Node_ConnListenEndpoint(outConns[bestIdx], &ep)) {
discovered_peer_t* p = Discovery_FindPtr(disc, &ep);
if (p) p->lastQueryMs = now;
}
toQuery[toQueryCount++] = outConns[bestIdx];
queries++;
}
}
// 5. Connect: pick REACHABLE, not-currently-connected, cooled-down peers with the lowest ping
// until we reach the target connection count.
if (outCount < (size_t)DISCOVERY_TARGET_CONNECTIONS) {
size_t slots = (size_t)DISCOVERY_TARGET_CONNECTIONS - outCount;
for (size_t s = 0; s < slots && toConnectCount < MAX_CONS; ++s) {
discovered_peer_t* best = NULL;
size_t n = vector_size(disc->peers);
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)vector_get(disc->peers, i);
if (!p || p->state != DISCOVERY_STATE_REACHABLE) continue;
if (!Discovery_ConnectCooledDown(disc, &p->addr, now)) continue;
int already = 0;
for (size_t j = 0; j < outEpCount; ++j) {
if (Discovery_AddrEqual(&p->addr, &outEndpoints[j])) { already = 1; break; }
}
// Skip other addresses of a node we already have an outbound connection to. Only
// outbound counts: an inbound connection from a peer is its own dial, and we still
// want one of our own to it (broadcasts only travel outbound).
if (!already && p->nodeId != 0) {
for (size_t j = 0; j < outEpCount; ++j) {
if (outNodeIds[j] == p->nodeId) { already = 1; break; }
}
}
if (already) continue;
if (!best || p->pingMs < best->pingMs) best = p;
}
if (!best) break;
Discovery_NoteConnectAttempt(disc, &best->addr, now); // reserve so it isn't picked again this tick
char ip[INET6_ADDRSTRLEN];
unsigned short port = 0;
if (Discovery_AddrToIpPort(&best->addr, ip, sizeof(ip), &port) && port != 0) {
strncpy(toConnect[toConnectCount].ip, ip, INET6_ADDRSTRLEN - 1);
toConnect[toConnectCount].ip[INET6_ADDRSTRLEN - 1] = '\0';
toConnect[toConnectCount].port = port;
toConnectCount++;
}
}
}
pthread_mutex_unlock(&disc->lock);
// Execute the deferred network actions outside the lock.
for (size_t i = 0; i < toQueryCount; ++i) {
Node_SendPacket(disc->node, toQuery[i], PACKET_TYPE_GET_PEERS, NULL, 0);
}
for (size_t i = 0; i < toConnectCount; ++i) {
printf("NodeDiscovery: connecting to discovered peer %s:%u\n", toConnect[i].ip, toConnect[i].port);
(void)Node_ConnectPeer(disc->node, toConnect[i].ip, toConnect[i].port);
}
}
// ---- diagnostics -------------------------------------------------------------------------
void NodeDiscovery_PrintPeers(node_discovery_t* disc) {
if (!disc) {
printf("NodeDiscovery: not active\n");
return;
}
static const char* stateNames[] = { "NEW", "PINGED", "REACHABLE", "CONNECTED", "UNREACHABLE" };
pthread_mutex_lock(&disc->lock);
size_t n = vector_size(disc->peers);
printf("Known peers (%zu):\n", n);
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)vector_get(disc->peers, i);
if (!p) continue;
char ip[INET6_ADDRSTRLEN] = {0};
unsigned short port = 0;
Discovery_AddrToIpPort(&p->addr, ip, sizeof(ip), &port);
const char* stateStr = (p->state <= DISCOVERY_STATE_UNREACHABLE) ? stateNames[p->state] : "?";
char idStr[19];
if (p->nodeId != 0) {
snprintf(idStr, sizeof(idStr), "%016" PRIx64, p->nodeId);
} else {
snprintf(idStr, sizeof(idStr), "%-16s", "?");
}
if (p->pingMs == UINT64_MAX) {
printf(" %-46s hop=%u state=%-11s id=%s ping=--\n", ip, p->hop, stateStr, idStr);
} else {
printf(" %-46s hop=%u state=%-11s id=%s ping=%" PRIu64 "ms\n", ip, p->hop, stateStr, idStr, p->pingMs);
}
(void)port; // port is part of ip endpoint identity; shown via connect logs
}
size_t selfCount = vector_size(disc->selfEndpoints);
printf("Own endpoints (%zu):\n", selfCount);
for (size_t i = 0; i < selfCount; ++i) {
const struct sockaddr_storage* self = (const struct sockaddr_storage*)vector_get_const(disc->selfEndpoints, i);
if (!self) continue;
char ip[INET6_ADDRSTRLEN] = {0};
unsigned short port = 0;
Discovery_AddrToIpPort(self, ip, sizeof(ip), &port);
printf(" %-46s port=%u\n", ip, port);
}
pthread_mutex_unlock(&disc->lock);
}
src/nets/orphan_pool.c
+520 -144
View File
@@ -1,5 +1,7 @@
#include <nets/orphan_pool.h> #include <nets/orphan_pool.h>
#include <dynarr.h> #include <constants.h>
#include <dlibc/vector.h>
#include <pthread.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#include <stdio.h> #include <stdio.h>
@@ -7,202 +9,576 @@
typedef struct { typedef struct {
block_t* block; block_t* block;
uint64_t height; uint64_t height;
uint64_t observedAtTipHeight; // local tip height when first seen; stamped once (reorg penalty)
uint64_t sequence; // insertion order, used to evict the oldest entry when full
uint8_t hash[32];
} orphan_entry_t; } orphan_entry_t;
static DynArr* g_orphans = NULL; static vector_t* g_orphans = NULL;
static uint64_t g_nextSequence = 0;
// The pool is touched by the maintenance thread, by every per-peer TCP thread and by the REPL
// thread. It used to have no synchronisation at all, so a concurrent Insert could realloc the
// array out from under a scan that was holding a raw element pointer.
//
// Lock ordering: this mutex is never held while calling into chain.c (which takes g_chainLock).
// Candidate branches are collected under the lock, the lock is dropped, and only then is
// Chain_ReplaceBranch/Chain_AddBlock called.
static pthread_mutex_t g_orphanLock = PTHREAD_MUTEX_INITIALIZER;
// Element destructor for `g_orphans`: an entry owns its block, so dropping or evicting an entry
// frees the block along with it. OrphanPool_TakeByHashLocked hands a block out through
// vector_take_at(), which deliberately skips this. Block_Destroy accepts NULL.
static void OrphanPool_EntryDestructor(void* element) {
orphan_entry_t* e = (orphan_entry_t*)element;
Block_Destroy(e->block);
}
static void OrphanPool_InitLocked(void) {
if (!g_orphans) {
g_orphans = vector_create(sizeof(orphan_entry_t));
vector_set_destructor(g_orphans, OrphanPool_EntryDestructor);
}
}
void OrphanPool_Init(void) { void OrphanPool_Init(void) {
if (g_orphans) return; pthread_mutex_lock(&g_orphanLock);
g_orphans = DYNARR_CREATE(orphan_entry_t, 16); OrphanPool_InitLocked();
pthread_mutex_unlock(&g_orphanLock);
} }
void OrphanPool_Destroy(void) { void OrphanPool_Destroy(void) {
if (!g_orphans) return; pthread_mutex_lock(&g_orphanLock);
size_t n = DynArr_size(g_orphans); vector_destroy(&g_orphans); // The element destructor frees every pooled block
for (size_t i = 0; i < n; ++i) { pthread_mutex_unlock(&g_orphanLock);
orphan_entry_t* e = (orphan_entry_t*)DynArr_at(g_orphans, i);
if (e && e->block) {
Block_Destroy(e->block);
}
}
DynArr_destroy(g_orphans);
g_orphans = NULL;
} }
void OrphanPool_Insert(block_t* block, uint64_t height) { static ssize_t OrphanPool_FindByHashLocked(const uint8_t blockHash[32]) {
if (!block) return; if (!g_orphans || !blockHash) {
if (!g_orphans) OrphanPool_Init(); return -1;
}
size_t n = vector_size(g_orphans);
for (size_t i = 0; i < n; ++i) {
orphan_entry_t* e = (orphan_entry_t*)vector_get(g_orphans, i);
if (e && memcmp(e->hash, blockHash, 32) == 0) {
return (ssize_t)i;
}
}
return -1;
}
// Drop the entry with the lowest sequence number, so a flood of unusable orphans cannot grow
// without bound. Returns true if something was evicted.
static bool OrphanPool_EvictOldestLocked(void) {
if (!g_orphans) {
return false;
}
size_t n = vector_size(g_orphans);
if (n == 0) {
return false;
}
size_t oldestIndex = 0;
uint64_t oldestSequence = UINT64_MAX;
for (size_t i = 0; i < n; ++i) {
orphan_entry_t* e = (orphan_entry_t*)vector_get(g_orphans, i);
if (e && e->sequence < oldestSequence) {
oldestSequence = e->sequence;
oldestIndex = i;
}
}
vector_pop_at(g_orphans, oldestIndex); // The element destructor frees the victim's block
return true;
}
void OrphanPool_Insert(block_t* block, uint64_t height, uint64_t observedAtTipHeight) {
if (!block) {
return;
}
uint8_t blockHash[32];
Block_CalculateHash(block, blockHash);
pthread_mutex_lock(&g_orphanLock);
OrphanPool_InitLocked();
if (!g_orphans) {
pthread_mutex_unlock(&g_orphanLock);
Block_Destroy(block);
return;
}
// Reject duplicates. The same block reaches us from every peer that relays it, and without
// this each copy became its own permanently-resident entry.
if (OrphanPool_FindByHashLocked(blockHash) >= 0) {
pthread_mutex_unlock(&g_orphanLock);
Block_Destroy(block);
return;
}
while (vector_size(g_orphans) >= MAX_ORPHAN_BLOCKS) {
if (!OrphanPool_EvictOldestLocked()) {
break;
}
}
orphan_entry_t e; orphan_entry_t e;
memset(&e, 0, sizeof(e));
e.block = block; e.block = block;
e.height = height; e.height = height;
(void)DynArr_push_back(g_orphans, &e); e.observedAtTipHeight = observedAtTipHeight;
e.sequence = g_nextSequence++;
memcpy(e.hash, blockHash, 32);
if (vector_push_back(g_orphans, &e) != 0) {
pthread_mutex_unlock(&g_orphanLock);
Block_Destroy(block); // never pooled, so the pool does not own it
return;
}
pthread_mutex_unlock(&g_orphanLock);
} }
static size_t OrphanPool_TryAdoptBranch(blockchain_t* chain, uint64_t forkHeight) { bool OrphanPool_Contains(const uint8_t blockHash[32]) {
if (!g_orphans || !chain) return 0; pthread_mutex_lock(&g_orphanLock);
bool found = OrphanPool_FindByHashLocked(blockHash) >= 0;
pthread_mutex_unlock(&g_orphanLock);
return found;
}
DynArr* seq = DYNARR_CREATE(block_t*, 8); size_t OrphanPool_Size(void) {
if (!seq) return 0; pthread_mutex_lock(&g_orphanLock);
size_t n = vector_size(g_orphans);
pthread_mutex_unlock(&g_orphanLock);
return n;
}
// Remove the entry with this hash without freeing the block, and hand the block back. Used once a
// block has been given to the chain, which then owns its transaction array. vector_take_at() skips
// the element destructor, which is what moves ownership of the block to the caller.
static block_t* OrphanPool_TakeByHashLocked(const uint8_t blockHash[32]) {
ssize_t index = OrphanPool_FindByHashLocked(blockHash);
if (index < 0) {
return NULL;
}
orphan_entry_t entry;
if (vector_take_at(g_orphans, (size_t)index, &entry) != 0) {
return NULL;
}
return entry.block;
}
static void OrphanPool_DropByHashLocked(const uint8_t blockHash[32]) {
ssize_t index = OrphanPool_FindByHashLocked(blockHash);
if (index < 0) {
return;
}
vector_pop_at(g_orphans, (size_t)index); // The element destructor frees the block
}
/**
* Copy out the orphan that extends `prevHash` at `height`, if any.
* Returns false when there is no such orphan. Caller must hold the pool lock.
**/
static bool OrphanPool_FindChildLocked(uint64_t height,
const uint8_t prevHash[32],
block_t** outBlock,
uint64_t* outObservedAtTipHeight,
uint8_t outHash[32]) {
if (!g_orphans) {
return false;
}
size_t n = vector_size(g_orphans);
for (size_t i = 0; i < n; ++i) {
orphan_entry_t* e = (orphan_entry_t*)vector_get(g_orphans, i);
if (!e || !e->block) {
continue;
}
if (e->height != height) {
continue;
}
if (memcmp(e->block->header.prevHash, prevHash, 32) != 0) {
continue;
}
*outBlock = e->block;
*outObservedAtTipHeight = e->observedAtTipHeight;
memcpy(outHash, e->hash, 32);
return true;
}
return false;
}
/**
* Follow prevHash links from `forkHeight` to build the longest branch the pool can offer.
*
* The old implementation took the first orphan found at each successive height with no linkage
* check at all, which could splice blocks from two different forks into one incoherent branch.
* Caller must hold the pool lock. The returned array borrows the pooled block pointers; the pool
* still owns them (Chain_ReplaceBranch applies copies).
**/
static size_t OrphanPool_CollectBranchLocked(uint64_t forkHeight,
const uint8_t forkParentHash[32],
block_t*** outBlocks,
uint8_t** outHashes,
uint64_t* outObservedAtTipHeight) {
*outBlocks = NULL;
*outHashes = NULL;
*outObservedAtTipHeight = 0;
vector_t* collected = vector_create(sizeof(block_t*));
vector_t* hashes = vector_create(32); // one whole 32-byte block hash per element, index-aligned with collected
if (!collected || !hashes) {
vector_destroy(&collected);
vector_destroy(&hashes);
return 0;
}
uint8_t expectedPrevHash[32];
memcpy(expectedPrevHash, forkParentHash, 32);
uint64_t earliestObserved = UINT64_MAX;
uint64_t cursor = forkHeight;
size_t count = 0;
size_t cursor = forkHeight;
while (1) { while (1) {
bool found = false; block_t* child = NULL;
size_t count = DynArr_size(g_orphans); uint64_t observed = 0;
for (size_t i = 0; i < count; ++i) { uint8_t childHash[32];
orphan_entry_t* entry = (orphan_entry_t*)DynArr_at(g_orphans, i); if (!OrphanPool_FindChildLocked(cursor, expectedPrevHash, &child, &observed, childHash)) {
if (!entry || !entry->block) continue;
if (entry->height == cursor) {
(void)DynArr_push_back(seq, &entry->block);
found = true;
break; break;
} }
// A block only counts once both pushes have landed, so the two vectors never drift out of
// step: the copy-out below reads index i from both.
if (vector_push_back(collected, &child) != 0) {
break;
} }
if (!found) break; if (vector_push_back(hashes, childHash) != 0) {
break;
}
if (observed < earliestObserved) {
earliestObserved = observed;
}
memcpy(expectedPrevHash, childHash, 32);
cursor++; cursor++;
count++;
} }
size_t seqCount = DynArr_size(seq); if (count == 0) {
if (seqCount == 0) { vector_destroy(&collected);
DynArr_destroy(seq); vector_destroy(&hashes);
return 0; return 0;
} }
size_t currentTipHeight = Chain_Size(chain) == 0 ? 0 : Chain_Size(chain) - 1; block_t** blocks = (block_t**)calloc(count, sizeof(block_t*));
size_t seqTopHeight = forkHeight + seqCount - 1; uint8_t* hashOut = (uint8_t*)calloc(count, 32);
if (seqTopHeight <= currentTipHeight) { if (!blocks || !hashOut) {
DynArr_destroy(seq); free(blocks);
free(hashOut);
vector_destroy(&collected);
vector_destroy(&hashes);
return 0; return 0;
} }
size_t rollbackHeight = (forkHeight == 0) ? 0 : (forkHeight - 1); for (size_t i = 0; i < count; ++i) {
if (!Chain_RollbackToHeight(chain, rollbackHeight)) { block_t** blockSlot = (block_t**)vector_get(collected, i);
DynArr_destroy(seq); const uint8_t* hashSlot = (const uint8_t*)vector_get_const(hashes, i);
return 0; if (!blockSlot || !hashSlot) {
count = i; // never hand back a branch with a hole in it
break;
}
blocks[i] = *blockSlot;
memcpy(&hashOut[i * 32], hashSlot, 32);
} }
vector_destroy(&collected);
vector_destroy(&hashes);
*outBlocks = blocks;
*outHashes = hashOut;
*outObservedAtTipHeight = earliestObserved == UINT64_MAX ? 0ULL : earliestObserved;
return count;
}
// Discard orphans that can no longer ever be applied: anything at or below the current tip whose
// hash does not match the block we actually have there. Without this the pool only ever grew, and
// permanently-invalid entries were retried on every 1 Hz maintenance tick.
static void OrphanPool_PruneStale(blockchain_t* chain) {
if (!chain) {
return;
}
const size_t chainSize = Chain_Size(chain);
// Collect the hashes to drop first, so we never call into chain.c while holding the pool lock.
vector_t* doomed = vector_create(32); // one whole 32-byte block hash per element
if (!doomed) {
return;
}
pthread_mutex_lock(&g_orphanLock);
size_t n = vector_size(g_orphans);
vector_t* candidates = vector_create(32); // index-aligned with candidateHeights
vector_t* candidateHeights = vector_create(sizeof(uint64_t));
if (candidates && candidateHeights) {
for (size_t i = 0; i < n; ++i) {
orphan_entry_t* e = (orphan_entry_t*)vector_get(g_orphans, i);
if (!e || !e->block) {
continue;
}
if (e->height >= (uint64_t)chainSize) {
continue; // still ahead of us; may attach later
}
if (vector_push_back(candidates, e->hash) != 0) {
break;
}
if (vector_push_back(candidateHeights, &e->height) != 0) {
vector_pop_back(candidates); // keep the two index-aligned
break;
}
}
}
pthread_mutex_unlock(&g_orphanLock);
size_t candidateCount = vector_size(candidateHeights);
for (size_t i = 0; i < candidateCount; ++i) {
const uint64_t* heightSlot = (const uint64_t*)vector_get_const(candidateHeights, i);
const uint8_t* orphanHash = (const uint8_t*)vector_get_const(candidates, i);
if (!heightSlot || !orphanHash) {
continue;
}
uint64_t height = *heightSlot;
block_t* local = NULL;
if (!Chain_GetBlockCopy(chain, (size_t)height, &local) || !local) {
continue;
}
uint8_t localHash[32];
Block_CalculateHash(local, localHash);
Block_Destroy(local);
// Same block we already have: pure duplicate, drop it. A different block at a height we
// have already passed is kept, because it may yet be the base of a heavier branch.
if (memcmp(localHash, orphanHash, 32) == 0) {
(void)vector_push_back(doomed, orphanHash);
}
}
size_t doomedCount = vector_size(doomed);
if (doomedCount > 0) {
pthread_mutex_lock(&g_orphanLock);
for (size_t i = 0; i < doomedCount; ++i) {
const uint8_t* h = (const uint8_t*)vector_get_const(doomed, i);
if (h) {
OrphanPool_DropByHashLocked(h);
}
}
pthread_mutex_unlock(&g_orphanLock);
}
vector_destroy(&candidates);
vector_destroy(&candidateHeights);
vector_destroy(&doomed);
}
/**
* Try to extend the current tip directly with pooled orphans.
* Returns the number of blocks attached.
**/
static size_t OrphanPool_ExtendTip(blockchain_t* chain) {
size_t attached = 0; size_t attached = 0;
for (size_t i = 0; i < seqCount; ++i) {
block_t* bptr = *(block_t**)DynArr_at(seq, i); while (1) {
if (!bptr || !Chain_AddBlock(chain, bptr)) { const size_t chainSize = Chain_Size(chain);
uint8_t tipHash[32];
memset(tipHash, 0, sizeof(tipHash));
if (chainSize > 0) {
block_t* tip = NULL;
if (!Chain_GetBlockCopy(chain, chainSize - 1, &tip) || !tip) {
break;
}
Block_CalculateHash(tip, tipHash);
Block_Destroy(tip);
}
// Take a copy of the candidate under the lock, then release it before touching the chain.
pthread_mutex_lock(&g_orphanLock);
block_t* pooled = NULL;
uint64_t observed = 0;
uint8_t candidateHash[32];
bool found = OrphanPool_FindChildLocked((uint64_t)chainSize, tipHash, &pooled, &observed, candidateHash);
block_t* candidate = found ? Block_Copy(pooled) : NULL;
pthread_mutex_unlock(&g_orphanLock);
if (!found) {
break;
}
if (!candidate) {
break; break;
} }
size_t count = DynArr_size(g_orphans); if (!Chain_AddBlock(chain, candidate)) {
for (size_t j = 0; j < count; ++j) { // Permanent rejection for this block at this height (bad coinbase, wrong difficulty,
orphan_entry_t* entry = (orphan_entry_t*)DynArr_at(g_orphans, j); // ...). Drop it rather than retrying it on every maintenance tick forever.
if (entry && entry->block == bptr) { Block_Destroy(candidate);
DynArr_remove(g_orphans, j); pthread_mutex_lock(&g_orphanLock);
break; OrphanPool_DropByHashLocked(candidateHash);
pthread_mutex_unlock(&g_orphanLock);
continue;
} }
// Chain_AddBlock took ownership of the transaction array and cleared our pointer to it.
Block_Destroy(candidate);
pthread_mutex_lock(&g_orphanLock);
block_t* taken = OrphanPool_TakeByHashLocked(candidateHash);
pthread_mutex_unlock(&g_orphanLock);
if (taken) {
Block_Destroy(taken); // the pool's own copy is independent of the one we applied
} }
attached++; attached++;
} }
DynArr_destroy(seq);
return attached; return attached;
} }
/**
* Look for a competing branch that forks below our tip and is worth adopting.
* The work comparison, the reorg penalty and the atomicity all live in Chain_ReplaceBranch.
**/
static size_t OrphanPool_TryAdoptBranch(blockchain_t* chain, bool bypassPenalty) {
const size_t chainSize = Chain_Size(chain);
if (chainSize == 0) {
return 0;
}
// Walk fork points from just below the tip downwards; the shallowest fork wins, which is also
// the one with the smallest reorg penalty.
for (size_t forkHeight = chainSize; forkHeight >= 1; --forkHeight) {
block_t* parent = NULL;
if (!Chain_GetBlockCopy(chain, forkHeight - 1, &parent) || !parent) {
continue;
}
uint8_t parentHash[32];
Block_CalculateHash(parent, parentHash);
Block_Destroy(parent);
pthread_mutex_lock(&g_orphanLock);
block_t** branch = NULL;
uint8_t* branchHashes = NULL;
uint64_t observedAtTipHeight = 0;
size_t branchCount = OrphanPool_CollectBranchLocked((uint64_t)forkHeight, parentHash,
&branch, &branchHashes, &observedAtTipHeight);
// Copy the branch so the pool lock can be released before we call into the chain.
block_t** branchCopies = NULL;
if (branchCount > 0) {
branchCopies = (block_t**)calloc(branchCount, sizeof(block_t*));
if (branchCopies) {
for (size_t i = 0; i < branchCount; ++i) {
branchCopies[i] = Block_Copy(branch[i]);
}
}
}
pthread_mutex_unlock(&g_orphanLock);
free(branch);
if (branchCount == 0 || !branchCopies) {
free(branchHashes);
if (branchCopies) {
free(branchCopies);
}
if (forkHeight == 1) break;
continue;
}
bool copiedAll = true;
for (size_t i = 0; i < branchCount; ++i) {
if (!branchCopies[i]) {
copiedAll = false;
}
}
bool adopted = false;
if (copiedAll) {
adopted = Chain_ReplaceBranch(chain, forkHeight, branchCopies, branchCount, observedAtTipHeight,
bypassPenalty);
}
for (size_t i = 0; i < branchCount; ++i) {
if (branchCopies[i]) {
Block_Destroy(branchCopies[i]); // Chain_ReplaceBranch applied its own copies
}
}
free(branchCopies);
if (adopted) {
printf("Adopted competing branch of %zu block(s) at fork height %zu\n", branchCount, forkHeight);
pthread_mutex_lock(&g_orphanLock);
for (size_t i = 0; i < branchCount; ++i) {
OrphanPool_DropByHashLocked(&branchHashes[i * 32]);
}
pthread_mutex_unlock(&g_orphanLock);
free(branchHashes);
return branchCount;
}
free(branchHashes);
if (forkHeight == 1) {
break;
}
}
return 0;
}
size_t OrphanPool_AttemptAttach(blockchain_t* chain) { size_t OrphanPool_AttemptAttach(blockchain_t* chain) {
if (!g_orphans || !chain) return 0; return OrphanPool_AttemptAttachForced(chain, false);
}
size_t OrphanPool_AttemptAttachForced(blockchain_t* chain, bool bypassPenalty) {
if (!chain) {
return 0;
}
pthread_mutex_lock(&g_orphanLock);
bool empty = vector_is_empty(g_orphans); // also true while the pool is not initialised
pthread_mutex_unlock(&g_orphanLock);
if (empty) {
return 0;
}
size_t attached = 0; size_t attached = 0;
bool madeProgress = true;
// Attempt repeatedly while progress is made (to handle chained orphans) // Extending the tip is always preferable to a reorg, so try that to exhaustion first, and only
while (madeProgress) { // then consider replacing part of our chain with a competing branch.
madeProgress = false; while (1) {
size_t n = DynArr_size(g_orphans); size_t extended = OrphanPool_ExtendTip(chain);
for (size_t i = 0; i < n; ++i) { attached += extended;
orphan_entry_t* e = (orphan_entry_t*)DynArr_at(g_orphans, i);
if (!e || !e->block) continue;
uint64_t parentIndex = (e->height == 0) ? (uint64_t)-1 : (e->height - 1); size_t adopted = OrphanPool_TryAdoptBranch(chain, bypassPenalty);
bool parentExists = false;
if (e->height == 0) {
// genesis-style block: parent is zero-hash; accept if chain empty
parentExists = (Chain_Size(chain) == 0);
} else if (parentIndex < Chain_Size(chain)) {
block_t* parent = NULL;
if (Chain_GetBlockCopy(chain, (size_t)parentIndex, &parent) && parent) {
parentExists = true;
Block_Destroy(parent);
} else {
parentExists = false;
}
}
if (parentExists) {
if (e->height < Chain_Size(chain)) {
block_t* local = NULL;
if (Chain_GetBlockCopy(chain, (size_t)e->height, &local) && local) {
uint8_t localHash[32];
uint8_t orphanHash[32];
Block_CalculateHash(local, localHash);
Block_CalculateHash(e->block, orphanHash);
Block_Destroy(local);
if (memcmp(localHash, orphanHash, 32) != 0) {
size_t adopted = OrphanPool_TryAdoptBranch(chain, e->height);
if (adopted > 0) {
attached += adopted; attached += adopted;
madeProgress = true;
n = DynArr_size(g_orphans); if (extended == 0 && adopted == 0) {
i = (size_t)-1;
break; break;
} }
} }
} else if (local) {
Block_Destroy(local);
}
}
// Verify that the parent's hash matches the orphan's prevHash before attaching. OrphanPool_PruneStale(chain);
bool parentMatches = false;
if (e->height == 0) {
parentMatches = (Chain_Size(chain) == 0);
} else {
block_t* parent = NULL;
if (Chain_GetBlockCopy(chain, (size_t)parentIndex, &parent) && parent) {
uint8_t parentHash[32];
Block_CalculateHash(parent, parentHash);
parentMatches = (memcmp(parentHash, e->block->header.prevHash, 32) == 0);
Block_Destroy(parent);
} else {
parentMatches = false;
}
}
if (!parentMatches) {
// Parent exists but does not match this orphan's prevHash.
size_t adopted = OrphanPool_TryAdoptBranch(chain, e->height);
if (adopted > 0) {
attached += adopted;
madeProgress = true;
n = DynArr_size(g_orphans);
i = (size_t)-1;
break;
}
continue;
}
// Try to add to chain
if (Chain_AddBlock(chain, e->block)) {
attached++;
madeProgress = true;
// remove this entry
DynArr_remove(g_orphans, i);
// adjust indices
n = DynArr_size(g_orphans);
i = (size_t)-1; // reset outer loop
break;
} else {
// Keep the orphan around; rejection may be temporary while the local tip is being reorged.
continue;
}
}
}
}
return attached; return attached;
} }
src/numgen.c
+31
View File
@@ -1,5 +1,8 @@
#include <numgen.h> #include <numgen.h>
#include <stdio.h>
#include <unistd.h>
unsigned char random_byte(void) { unsigned char random_byte(void) {
return (unsigned char)(rand() % 256); return (unsigned char)(rand() % 256);
} }
@@ -39,3 +42,31 @@ uint64_t random_eight_byte(void) {
return x; return x;
} }
uint64_t random_secure_eight_byte(void) {
uint64_t x = 0;
FILE* urandom = fopen("/dev/urandom", "rb");
if (urandom) {
size_t got = fread(&x, 1, sizeof(x), urandom);
fclose(urandom);
if (got == sizeof(x) && x != 0) {
return x;
}
}
// Fallback: srand() is seeded from the wall clock in whole seconds, so two nodes launched
// together would draw identical values. Mix in the pid and the sub-second clock to separate them.
struct timespec ts;
if (clock_gettime(CLOCK_REALTIME, &ts) != 0) {
ts.tv_sec = 0;
ts.tv_nsec = 0;
}
x = random_eight_byte();
x ^= (uint64_t)ts.tv_nsec;
x ^= ((uint64_t)ts.tv_sec) << 16;
x ^= ((uint64_t)getpid()) << 40;
return x ? x : 1; // 0 means "no identity advertised" on the wire
}
src/tcpd/tcpconnection.c
+15
View File
@@ -31,6 +31,7 @@ int TcpConnection_Init(tcp_connection_t* conn, int sockFd, const struct sockaddr
conn->closing = false; conn->closing = false;
conn->disconnectedNotified = false; conn->disconnectedNotified = false;
atomic_init(&conn->pinCount, 0);
conn->dataBuf = NULL; conn->dataBuf = NULL;
conn->dataBufLen = 0; conn->dataBufLen = 0;
conn->dataBufCap = 0; conn->dataBufCap = 0;
@@ -262,6 +263,20 @@ bool TcpConnection_IsDisconnectNotified(tcp_connection_t* conn) {
return notified; return notified;
} }
void TcpConnection_Pin(tcp_connection_t* conn) {
if (!conn) {
return;
}
atomic_fetch_add(&conn->pinCount, 1);
}
void TcpConnection_Unpin(tcp_connection_t* conn) {
if (!conn) {
return;
}
atomic_fetch_sub(&conn->pinCount, 1);
}
static int extract_v4(const tcp_connection_t* conn, struct in_addr* v4out) { static int extract_v4(const tcp_connection_t* conn, struct in_addr* v4out) {
if (conn->addrFamily == AF_INET6) { if (conn->addrFamily == AF_INET6) {
const struct sockaddr_in6* a6 = (const struct sockaddr_in6*)&conn->peerAddr; const struct sockaddr_in6* a6 = (const struct sockaddr_in6*)&conn->peerAddr;
src/tcpd/tcpserver.c
+81 -15
View File
@@ -16,15 +16,20 @@ typedef struct {
int listenFd; int listenFd;
} tcpaccept_thread_args_t; } tcpaccept_thread_args_t;
static void TcpServer_RemoveClientByPtrUnlocked(tcp_server_t* svr, tcp_connection_t* cli) { // Returns non-zero if `cli` was still registered (and has now been unregistered). A zero return
// means someone else already claimed the slot -- see the detach logic in the client thread.
static int TcpServer_RemoveClientByPtrUnlocked(tcp_server_t* svr, tcp_connection_t* cli) {
if (!svr || !svr->clientsArrPtr || !cli) { if (!svr || !svr->clientsArrPtr || !cli) {
return; return 0;
} }
size_t idx = Generic_FindClientInArrayByPtr(svr->clientsArrPtr, cli, svr->maxClients); size_t idx = Generic_FindClientInArrayByPtr(svr->clientsArrPtr, cli, svr->maxClients);
if (idx != SIZE_MAX) { if (idx != SIZE_MAX) {
svr->clientsArrPtr[idx] = NULL; svr->clientsArrPtr[idx] = NULL;
return 1;
} }
return 0;
} }
static void* TcpServer_clientthreadprocess(void* ptr) { static void* TcpServer_clientthreadprocess(void* ptr) {
@@ -65,16 +70,38 @@ static void* TcpServer_clientthreadprocess(void* ptr) {
cli->on_disconnect(cli); cli->on_disconnect(cli);
} }
// Unregister, decide who joins us, and free -- all under clientsMutex.
//
// The destroy/free used to happen after the lock was released, which left a window where
// TcpServer_Stop could be holding this very pointer and about to use it. Doing it under the
// same lock Stop uses to inspect the slots removes that window entirely.
pthread_mutex_lock(&svr->clientsMutex); pthread_mutex_lock(&svr->clientsMutex);
TcpServer_RemoveClientByPtrUnlocked(svr, cli);
pthread_mutex_unlock(&svr->clientsMutex); // If our slot was still ours, TcpServer_Stop has not claimed us and never will (we are leaving
// the array now), so nobody is going to join this thread -- detach it or its resources leak.
// If the slot was already cleared, Stop took our handle and is waiting in pthread_join, so we
// must stay joinable.
if (TcpServer_RemoveClientByPtrUnlocked(svr, cli)) {
pthread_detach(pthread_self());
}
TcpConnection_Destroy(cli); TcpConnection_Destroy(cli);
free(cli); free(cli);
pthread_mutex_unlock(&svr->clientsMutex);
return NULL; return NULL;
} }
// listenFd is borrowed, not owned: it is ptr->sockFd / ptr->sockFdV4, handed
// over by TcpServer_Start and closed by TcpServer_Stop. GCC's -fanalyzer infers
// from accept() that the fd is open and then holds this function responsible
// for closing it, so it reports a leak on every path that leaves the loop.
// Clang does not know this warning group, hence the guard.
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wanalyzer-fd-leak"
#endif
static void* TcpServer_threadprocess(void* ptr) { static void* TcpServer_threadprocess(void* ptr) {
tcpaccept_thread_args_t* args = (tcpaccept_thread_args_t*)ptr; tcpaccept_thread_args_t* args = (tcpaccept_thread_args_t*)ptr;
if (!args || !args->serverPtr) { if (!args || !args->serverPtr) {
@@ -170,6 +197,9 @@ static void* TcpServer_threadprocess(void* ptr) {
return NULL; return NULL;
} }
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic pop
#endif
tcp_server_t* TcpServer_Create() { tcp_server_t* TcpServer_Create() {
tcp_server_t* svr = (tcp_server_t*)malloc(sizeof(*svr)); tcp_server_t* svr = (tcp_server_t*)malloc(sizeof(*svr));
@@ -208,6 +238,16 @@ void TcpServer_Destroy(tcp_server_t* ptr) {
free(ptr); free(ptr);
} }
// Both sockets are handed to the caller through ptr->sockFd / ptr->sockFdV4 and
// closed by TcpServer_Stop, so neither leaks. -fanalyzer loses track of the
// first store across the second socket's branches and reports it anyway; the
// report is positional, not semantic — swapping the IPv6 and IPv4 blocks moves
// the warning from fd6 to fd4, and deleting the unrelated second block silences
// it entirely.
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wanalyzer-fd-leak"
#endif
void TcpServer_Init(tcp_server_t* ptr, unsigned short port, const char* addr) { void TcpServer_Init(tcp_server_t* ptr, unsigned short port, const char* addr) {
if (!ptr || !addr) { if (!ptr || !addr) {
return; return;
@@ -255,7 +295,18 @@ void TcpServer_Init(tcp_server_t* ptr, unsigned short port, const char* addr) {
} }
} }
} }
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic pop
#endif
// Same borrowed-fd false positive as TcpServer_threadprocess: listen() teaches
// -fanalyzer that ptr->sockFd / ptr->sockFdV4 are open passive sockets, so it
// expects this function to close them. They belong to the tcp_server_t and are
// closed by TcpServer_Stop.
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wanalyzer-fd-leak"
#endif
void TcpServer_Start(tcp_server_t* ptr, int maxcons) { void TcpServer_Start(tcp_server_t* ptr, int maxcons) {
if (!ptr || (ptr->sockFd < 0 && ptr->sockFdV4 < 0) || maxcons <= 0 || ptr->isRunning) { if (!ptr || (ptr->sockFd < 0 && ptr->sockFdV4 < 0) || maxcons <= 0 || ptr->isRunning) {
return; return;
@@ -329,6 +380,9 @@ void TcpServer_Start(tcp_server_t* ptr, int maxcons) {
pthread_mutex_unlock(&ptr->clientsMutex); pthread_mutex_unlock(&ptr->clientsMutex);
} }
} }
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic pop
#endif
void TcpServer_Stop(tcp_server_t* ptr) { void TcpServer_Stop(tcp_server_t* ptr) {
if (!ptr || !ptr->isRunning) { if (!ptr || !ptr->isRunning) {
@@ -359,30 +413,42 @@ void TcpServer_Stop(tcp_server_t* ptr) {
} }
ptr->svrThreadV4 = 0; ptr->svrThreadV4 = 0;
// Ask every live client to close and copy out its thread handle, all under clientsMutex.
//
// This used to read the client slots with the lock released, which races with an exiting client
// thread clearing its own slot -- and worse, that thread destroys and frees the connection right
// afterwards, so the pointer read here could already be freed memory. Copying the pthread_t
// while holding the lock means the join below never dereferences the connection at all, and the
// client thread cannot free itself out from under us because it does that under the same lock.
pthread_mutex_lock(&ptr->clientsMutex); pthread_mutex_lock(&ptr->clientsMutex);
size_t maxClients = ptr->maxClients; size_t maxClients = ptr->maxClients;
tcp_connection_t** local = ptr->clientsArrPtr; pthread_t* joinHandles = maxClients ? (pthread_t*)calloc(maxClients, sizeof(pthread_t)) : NULL;
pthread_mutex_unlock(&ptr->clientsMutex); size_t joinCount = 0;
if (ptr->clientsArrPtr) {
for (size_t i = 0; i < maxClients; ++i) { for (size_t i = 0; i < maxClients; ++i) {
tcp_connection_t* cli = local[i]; tcp_connection_t* cli = ptr->clientsArrPtr[i];
if (!cli) { if (!cli) {
continue; continue;
} }
TcpConnection_RequestClose(cli); TcpConnection_RequestClose(cli);
}
for (size_t i = 0; i < maxClients; ++i) { if (joinHandles && !pthread_equal(cli->ioThread, pthread_self())) {
tcp_connection_t* cli = local[i]; joinHandles[joinCount++] = cli->ioThread;
if (!cli) { // Claim the slot: the client thread checks whether it is still registered to decide
continue; // whether to detach itself or stay joinable for the pthread_join below.
ptr->clientsArrPtr[i] = NULL;
} }
}
}
pthread_mutex_unlock(&ptr->clientsMutex);
if (!pthread_equal(cli->ioThread, pthread_self())) { // Join outside the lock: a client thread needs clientsMutex to finish unregistering itself.
pthread_join(cli->ioThread, NULL); for (size_t i = 0; i < joinCount; ++i) {
} pthread_join(joinHandles[i], NULL);
} }
free(joinHandles);
pthread_mutex_lock(&ptr->clientsMutex); pthread_mutex_lock(&ptr->clientsMutex);
free(ptr->clientsArrPtr); free(ptr->clientsArrPtr);
src/txmempool.c
+74 -27
View File
@@ -1,19 +1,66 @@
#include <txmempool.h> #include <txmempool.h>
#include <constants.h>
#include <pthread.h> #include <pthread.h>
static pthread_mutex_t g_txMempoolLock; static pthread_mutex_t g_txMempoolLock;
static bool g_txMempoolLockInitialized = false; static bool g_txMempoolLockInitialized = false;
khash_t(tx_mempool_map_m)* txMempool = NULL; khash_t(tx_mempool_map_m)* g_txMempool = NULL;
void TxMempool_Init() { void TxMempool_Init() {
txMempool = kh_init(tx_mempool_map_m); g_txMempool = kh_init(tx_mempool_map_m);
pthread_mutex_init(&g_txMempoolLock, NULL); pthread_mutex_init(&g_txMempoolLock, NULL);
g_txMempoolLockInitialized = true; g_txMempoolLockInitialized = true;
} }
bool TxMempool_PolicyAccepts(const signed_transaction_t* tx, uint64_t nowMs) {
if (!tx) {
return false;
}
const uint64_t ts = tx->transaction.timestamp;
// Dated too far in the future, measured against OUR CLOCK rather than the chain tip -- see the
// note in the header. Refusing this also limits the one real footgun in the replay guard: a
// wildly future timestamp permanently advances that account's lastTxTimestamp and locks it out
// until real time catches up.
if (ts > nowMs && (ts - nowMs) > TX_MAX_FUTURE_DRIFT_MS) {
return false;
}
// Too old to be worth holding. Not a validity judgement -- just pool hygiene.
if (nowMs > ts && (nowMs - ts) > TX_EXPIRY_MS) {
return false;
}
return true;
}
size_t TxMempool_PruneExpired(uint64_t nowMs) {
if (!g_txMempool) {
return 0;
}
size_t removed = 0;
pthread_mutex_lock(&g_txMempoolLock);
for (khiter_t k = kh_begin(g_txMempool); k != kh_end(g_txMempool); ++k) {
if (!kh_exist(g_txMempool, k)) {
continue;
}
const uint64_t ts = kh_value(g_txMempool, k).transaction.timestamp;
if (nowMs > ts && (nowMs - ts) > TX_EXPIRY_MS) {
kh_del(tx_mempool_map_m, g_txMempool, k);
removed++;
}
}
pthread_mutex_unlock(&g_txMempoolLock);
return removed;
}
int TxMempool_Insert(signed_transaction_t tx) { int TxMempool_Insert(signed_transaction_t tx) {
if (!txMempool) { return -1; } if (!g_txMempool) { return -1; }
pthread_mutex_lock(&g_txMempoolLock); pthread_mutex_lock(&g_txMempoolLock);
uint8_t txHash[32]; uint8_t txHash[32];
@@ -23,13 +70,13 @@ int TxMempool_Insert(signed_transaction_t tx) {
memcpy(key.bytes, txHash, 32); memcpy(key.bytes, txHash, 32);
int ret; int ret;
khiter_t k = kh_put(tx_mempool_map_m, txMempool, key, &ret); khiter_t k = kh_put(tx_mempool_map_m, g_txMempool, key, &ret);
if (k == kh_end(txMempool)) { if (k == kh_end(g_txMempool)) {
pthread_mutex_unlock(&g_txMempoolLock); pthread_mutex_unlock(&g_txMempoolLock);
return -1; return -1;
} }
kh_value(txMempool, k) = tx; kh_value(g_txMempool, k) = tx;
pthread_mutex_unlock(&g_txMempoolLock); pthread_mutex_unlock(&g_txMempoolLock);
@@ -37,15 +84,15 @@ int TxMempool_Insert(signed_transaction_t tx) {
} }
bool TxMempool_Lookup(uint8_t* txHash, signed_transaction_t* out) { bool TxMempool_Lookup(uint8_t* txHash, signed_transaction_t* out) {
if (!txMempool || !txHash || !out) { return false; } if (!g_txMempool || !txHash || !out) { return false; }
pthread_mutex_lock(&g_txMempoolLock); pthread_mutex_lock(&g_txMempoolLock);
key32_t key; key32_t key;
memcpy(key.bytes, txHash, 32); memcpy(key.bytes, txHash, 32);
khiter_t k = kh_get(tx_mempool_map_m, txMempool, key); khiter_t k = kh_get(tx_mempool_map_m, g_txMempool, key);
if (k != kh_end(txMempool)) { if (k != kh_end(g_txMempool)) {
signed_transaction_t tx = kh_value(txMempool, k); signed_transaction_t tx = kh_value(g_txMempool, k);
memcpy(out, &tx, sizeof(signed_transaction_t)); memcpy(out, &tx, sizeof(signed_transaction_t));
pthread_mutex_unlock(&g_txMempoolLock); pthread_mutex_unlock(&g_txMempoolLock);
return true; return true;
@@ -63,7 +110,7 @@ bool TxMempool_Snapshot(signed_transaction_t** outTxs, size_t* outCount) {
*outTxs = NULL; *outTxs = NULL;
*outCount = 0; *outCount = 0;
if (!txMempool) { if (!g_txMempool) {
return true; return true;
} }
@@ -71,8 +118,8 @@ bool TxMempool_Snapshot(signed_transaction_t** outTxs, size_t* outCount) {
size_t count = 0; size_t count = 0;
khiter_t k; khiter_t k;
for (k = kh_begin(txMempool); k != kh_end(txMempool); ++k) { for (k = kh_begin(g_txMempool); k != kh_end(g_txMempool); ++k) {
if (kh_exist(txMempool, k)) { if (kh_exist(g_txMempool, k)) {
++count; ++count;
} }
} }
@@ -89,9 +136,9 @@ bool TxMempool_Snapshot(signed_transaction_t** outTxs, size_t* outCount) {
} }
size_t index = 0; size_t index = 0;
for (k = kh_begin(txMempool); k != kh_end(txMempool); ++k) { for (k = kh_begin(g_txMempool); k != kh_end(g_txMempool); ++k) {
if (kh_exist(txMempool, k)) { if (kh_exist(g_txMempool, k)) {
snapshot[index++] = kh_value(txMempool, k); snapshot[index++] = kh_value(g_txMempool, k);
} }
} }
@@ -103,13 +150,13 @@ bool TxMempool_Snapshot(signed_transaction_t** outTxs, size_t* outCount) {
} }
void TxMempool_Print() { void TxMempool_Print() {
if (!txMempool) { return; } if (!g_txMempool) { return; }
pthread_mutex_lock(&g_txMempoolLock); pthread_mutex_lock(&g_txMempoolLock);
khiter_t k; khiter_t k;
for (k = kh_begin(txMempool); k != kh_end(txMempool); ++k) { for (k = kh_begin(g_txMempool); k != kh_end(g_txMempool); ++k) {
if (kh_exist(txMempool, k)) { if (kh_exist(g_txMempool, k)) {
signed_transaction_t tx = kh_val(txMempool, k); signed_transaction_t tx = kh_val(g_txMempool, k);
char senderHex[65]; char senderHex[65];
char recipient1Hex[65]; char recipient1Hex[65];
char recipient2Hex[65]; char recipient2Hex[65];
@@ -127,10 +174,10 @@ void TxMempool_Print() {
} }
void TxMempool_Destroy() { void TxMempool_Destroy() {
if (txMempool) { if (g_txMempool) {
pthread_mutex_lock(&g_txMempoolLock); pthread_mutex_lock(&g_txMempoolLock);
kh_destroy(tx_mempool_map_m, txMempool); kh_destroy(tx_mempool_map_m, g_txMempool);
txMempool = NULL; g_txMempool = NULL;
pthread_mutex_unlock(&g_txMempoolLock); pthread_mutex_unlock(&g_txMempoolLock);
} }
@@ -141,19 +188,19 @@ void TxMempool_Destroy() {
} }
bool TxMempool_Remove(const uint8_t* txHash) { bool TxMempool_Remove(const uint8_t* txHash) {
if (!txMempool || !txHash) { return false; } if (!g_txMempool || !txHash) { return false; }
pthread_mutex_lock(&g_txMempoolLock); pthread_mutex_lock(&g_txMempoolLock);
key32_t key; key32_t key;
memcpy(key.bytes, txHash, 32); memcpy(key.bytes, txHash, 32);
khiter_t k = kh_get(tx_mempool_map_m, txMempool, key); khiter_t k = kh_get(tx_mempool_map_m, g_txMempool, key);
if (k == kh_end(txMempool)) { if (k == kh_end(g_txMempool)) {
pthread_mutex_unlock(&g_txMempoolLock); pthread_mutex_unlock(&g_txMempoolLock);
return false; return false;
} }
kh_del(tx_mempool_map_m, txMempool, k); kh_del(tx_mempool_map_m, g_txMempool, k);
pthread_mutex_unlock(&g_txMempoolLock); pthread_mutex_unlock(&g_txMempoolLock);
return true; return true;
} }
src/udpd/udpnode.c
+390
View File
@@ -0,0 +1,390 @@
#include <udpd/udpnode.h>
#include <udpd/udppackettype.h>
#include <utils.h>
#include <numgen.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <errno.h>
typedef struct {
udp_node_t* node;
int sockFd;
} udprecv_thread_args_t;
// Send a raw PING packet (nonce already chosen) to dest.
static void UdpNode_SendRawPing(udp_node_t* node, uint64_t nonce, const struct sockaddr_storage* dest) {
unsigned char buf[UDP_PING_WIRE_SIZE];
buf[0] = (unsigned char)UDP_PACKET_TYPE_PING;
memcpy(buf + 1, &nonce, sizeof(nonce));
int sock = -1;
socklen_t addrLen = 0;
if (dest->ss_family == AF_INET6 && node->sockFd >= 0) {
sock = node->sockFd;
addrLen = sizeof(struct sockaddr_in6);
} else if (dest->ss_family == AF_INET && node->sockFdV4 >= 0) {
sock = node->sockFdV4;
addrLen = sizeof(struct sockaddr_in);
}
if (sock < 0) {
return;
}
sendto(sock, buf, sizeof(buf), 0, (const struct sockaddr*)dest, addrLen);
}
static void UdpNode_HandlePacket(udp_node_t* node, int fromSock,
const unsigned char* buf, ssize_t n,
const struct sockaddr_storage* from) {
if (n < 1) {
return;
}
udp_packet_type_t type = (udp_packet_type_t)buf[0];
switch (type) {
case UDP_PACKET_TYPE_PING: {
if (n < UDP_PING_WIRE_SIZE) {
return;
}
uint64_t nonce;
memcpy(&nonce, buf + 1, sizeof(nonce));
// Build and send PONG
unsigned char reply[UDP_PONG_WIRE_SIZE];
reply[0] = (unsigned char)UDP_PACKET_TYPE_PONG;
memcpy(reply + 1, &nonce, sizeof(nonce));
int32_t protoVer = (int32_t)PROTO_VERSION;
memcpy(reply + 1 + sizeof(nonce), &protoVer, sizeof(protoVer));
socklen_t addrLen = (from->ss_family == AF_INET6)
? sizeof(struct sockaddr_in6)
: sizeof(struct sockaddr_in);
sendto(fromSock, reply, sizeof(reply), 0, (const struct sockaddr*)from, addrLen);
break;
}
case UDP_PACKET_TYPE_PONG: {
if (n < UDP_PONG_WIRE_SIZE) {
return;
}
uint64_t nonce;
int32_t protoVer;
memcpy(&nonce, buf + 1, sizeof(nonce));
memcpy(&protoVer, buf + 1 + sizeof(nonce), sizeof(protoVer));
bool found = false;
uint64_t rttMs = 0;
pthread_mutex_lock(&node->pingsMutex);
for (int i = 0; i < UDP_MAX_PENDING_PINGS; i++) {
if (node->pendingPings[i].active && node->pendingPings[i].nonce == nonce) {
uint64_t nowMs = get_current_time_ms();
rttMs = (nowMs >= node->pendingPings[i].lastSentMs)
? (nowMs - node->pendingPings[i].lastSentMs)
: 0;
node->pendingPings[i].active = false;
found = true;
break;
}
}
pthread_mutex_unlock(&node->pingsMutex);
if (found && node->on_pong) {
node->on_pong(node, from, nonce, (int)protoVer, rttMs, node->callbackUser);
}
break;
}
default:
break;
}
}
static void* UdpNode_RecvThreadProc(void* arg) {
udprecv_thread_args_t* args = (udprecv_thread_args_t*)arg;
udp_node_t* node = args->node;
int sock = args->sockFd;
free(args);
unsigned char buf[1500];
while (node->isRunning) {
struct sockaddr_storage from;
socklen_t fromLen = sizeof(from);
ssize_t n = recvfrom(sock, buf, sizeof(buf), 0,
(struct sockaddr*)&from, &fromLen);
if (n < 1) {
if (!node->isRunning) {
break;
}
// Transient error — keep going
continue;
}
UdpNode_HandlePacket(node, sock, buf, n, &from);
}
return NULL;
}
static void* UdpNode_RetryThreadProc(void* arg) {
udp_node_t* node = (udp_node_t*)arg;
struct {
uint64_t nonce;
struct sockaddr_storage dest;
} timedOut[UDP_MAX_PENDING_PINGS];
while (node->isRunning) {
sleep_for_milliseconds(100);
int timedOutCount = 0;
pthread_mutex_lock(&node->pingsMutex);
uint64_t now = get_current_time_ms();
for (int i = 0; i < UDP_MAX_PENDING_PINGS; i++) {
pending_ping_t* p = &node->pendingPings[i];
if (!p->active) {
continue;
}
if (now - p->lastSentMs < UDP_PING_RETRY_INTERVAL_MS) {
continue;
}
if (p->retries >= UDP_PING_MAX_RETRIES) {
timedOut[timedOutCount].nonce = p->nonce;
timedOut[timedOutCount].dest = p->dest;
timedOutCount++;
p->active = false;
} else {
UdpNode_SendRawPing(node, p->nonce, &p->dest);
p->retries++;
p->lastSentMs = now;
}
}
pthread_mutex_unlock(&node->pingsMutex);
for (int i = 0; i < timedOutCount; i++) {
if (node->on_ping_timeout) {
node->on_ping_timeout(node, &timedOut[i].dest,
timedOut[i].nonce, node->callbackUser);
}
}
}
return NULL;
}
// Same borrowed/escaped-fd false positive as TcpServer_Init: both sockets are
// handed to the caller through node->sockFd / node->sockFdV4 and closed by
// UdpNode_Stop. -fanalyzer loses the first store across the second socket's
// branches and reports it as a leak.
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wanalyzer-fd-leak"
#endif
int UdpNode_Init(udp_node_t* node, uint16_t port) {
if (!node) {
return -1;
}
memset(node, 0, sizeof(*node));
node->sockFd = -1;
node->sockFdV4 = -1;
int opt = 1;
// IPv6 (pure, not dual-stack)
int fd6 = socket(AF_INET6, SOCK_DGRAM, 0);
if (fd6 >= 0) {
setsockopt(fd6, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
int v6only = 1;
setsockopt(fd6, IPPROTO_IPV6, IPV6_V6ONLY, &v6only, sizeof(v6only));
struct sockaddr_in6 a6;
memset(&a6, 0, sizeof(a6));
a6.sin6_family = AF_INET6;
a6.sin6_port = htons(port);
a6.sin6_addr = in6addr_any;
if (bind(fd6, (struct sockaddr*)&a6, sizeof(a6)) == 0) {
node->sockFd = fd6;
} else {
close(fd6);
}
}
// IPv4
int fd4 = socket(AF_INET, SOCK_DGRAM, 0);
if (fd4 >= 0) {
setsockopt(fd4, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
struct sockaddr_in a4;
memset(&a4, 0, sizeof(a4));
a4.sin_family = AF_INET;
a4.sin_port = htons(port);
a4.sin_addr.s_addr = INADDR_ANY;
if (bind(fd4, (struct sockaddr*)&a4, sizeof(a4)) == 0) {
node->sockFdV4 = fd4;
} else {
close(fd4);
}
}
if (node->sockFd < 0 && node->sockFdV4 < 0) {
return -1;
}
pthread_mutex_init(&node->pingsMutex, NULL);
return 0;
}
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic pop
#endif
void UdpNode_SetCallbacks(udp_node_t* node,
void (*on_pong)(udp_node_t*, const struct sockaddr_storage*, uint64_t, int, uint64_t, void*),
void (*on_ping_timeout)(udp_node_t*, const struct sockaddr_storage*, uint64_t, void*),
void* user) {
if (!node) {
return;
}
node->on_pong = on_pong;
node->on_ping_timeout = on_ping_timeout;
node->callbackUser = user;
}
int UdpNode_Start(udp_node_t* node) {
if (!node || node->isRunning) {
return -1;
}
if (node->sockFd < 0 && node->sockFdV4 < 0) {
return -1;
}
node->isRunning = 1;
int anyStarted = 0;
if (node->sockFd >= 0) {
udprecv_thread_args_t* args = (udprecv_thread_args_t*)malloc(sizeof(*args));
if (args) {
args->node = node;
args->sockFd = node->sockFd;
if (pthread_create(&node->recvThreadV6, NULL, UdpNode_RecvThreadProc, args) == 0) {
anyStarted = 1;
} else {
free(args);
}
}
}
if (node->sockFdV4 >= 0) {
udprecv_thread_args_t* args = (udprecv_thread_args_t*)malloc(sizeof(*args));
if (args) {
args->node = node;
args->sockFd = node->sockFdV4;
if (pthread_create(&node->recvThreadV4, NULL, UdpNode_RecvThreadProc, args) == 0) {
anyStarted = 1;
} else {
free(args);
}
}
}
if (pthread_create(&node->retryThread, NULL, UdpNode_RetryThreadProc, node) == 0) {
anyStarted = 1;
}
if (!anyStarted) {
node->isRunning = 0;
return -1;
}
return 0;
}
void UdpNode_Stop(udp_node_t* node) {
if (!node || !node->isRunning) {
return;
}
node->isRunning = 0;
// Close sockets to unblock recvfrom in receive threads
if (node->sockFd >= 0) {
int fd = node->sockFd;
node->sockFd = -1;
close(fd);
}
if (node->sockFdV4 >= 0) {
int fd = node->sockFdV4;
node->sockFdV4 = -1;
close(fd);
}
pthread_join(node->recvThreadV6, NULL);
pthread_join(node->recvThreadV4, NULL);
pthread_join(node->retryThread, NULL);
}
void UdpNode_Destroy(udp_node_t* node) {
if (!node) {
return;
}
if (node->sockFd >= 0) {
close(node->sockFd);
node->sockFd = -1;
}
if (node->sockFdV4 >= 0) {
close(node->sockFdV4);
node->sockFdV4 = -1;
}
pthread_mutex_destroy(&node->pingsMutex);
}
int UdpNode_SendPing(udp_node_t* node, const struct sockaddr_storage* dest) {
if (!node || !dest) {
return -1;
}
if (node->sockFd < 0 && node->sockFdV4 < 0) {
return -1;
}
uint64_t nonce = random_eight_byte();
uint64_t now = get_current_time_ms();
pthread_mutex_lock(&node->pingsMutex);
int slot = -1;
for (int i = 0; i < UDP_MAX_PENDING_PINGS; i++) {
if (!node->pendingPings[i].active) {
slot = i;
break;
}
}
if (slot < 0) {
pthread_mutex_unlock(&node->pingsMutex);
return -1;
}
node->pendingPings[slot].nonce = nonce;
node->pendingPings[slot].dest = *dest;
node->pendingPings[slot].lastSentMs = now;
node->pendingPings[slot].retries = 0;
node->pendingPings[slot].active = true;
pthread_mutex_unlock(&node->pingsMutex);
UdpNode_SendRawPing(node, nonce, dest);
return 0;
}