AnT (Ant's not TeX)
===================

AnT is free software that makes repeated LaTeX builds faster.  Its command
is `ant'.  It needs no root access and changes neither the TeX engine nor
your document.

While TeX runs, AnT keeps frozen copies ("snapshots") of the running
engine: one just before \begin{document} and one before each \include.
After an edit, AnT continues from the last snapshot before the change, so
the preamble, its packages and fonts, and all unchanged chapters before
the edit are not processed again.

You give AnT the TeX command you would type without it, as with env or
time; AnT's own options come before it:

    ant lualatex thesis             draft build (default): fast,
                                    references not updated
    ant --final lualatex thesis     final build: references, citations
                                    and table of contents up to date
    ant -v pdflatex thesis          also show AnT's details (passes,
                                    snapshots, memory)
    ant pdflatex -synctex=1 thesis  TeX's options go to TeX unchanged
    ant -q thesis                   short for `ant lualatex thesis'

`man ant' has the full manual.


Contents
--------

  1. What you see
  2. Installation
  3. How it works
  4. Snapshots on disk
  5. Measured results
  6. Building without snapshots
  7. Options and environment
  8. Limitations
  9. Files in this package
 10. Copyright and license


1. What you see
---------------

TeX's own terminal output, followed by AnT's messages:

    Output written on talk.pdf (14 pages, 74150 bytes).
    Transcript written on talk.log.
    ant: 1.8s instead of ~5.6s for a plain run; saved 3.8s (68% less time).

Each message is one line and starts with `ant:', as with other Unix
commands; --silent (or ANT_SILENT) leaves out all but error messages.  The
comparison uses the time of the last full run of the document.  When a
build saves no time, AnT says so and gives the reason.  Every message
AnT can print is listed, with examples, in the file MESSAGES; all texts
live in src/ant/messages.py.

AnT runs your command as you typed it, in the current folder, and adds
only -recorder right after the command name: it needs the list of files
TeX reads, which TeX writes to <job>.fls.  (XeTeX also gets -no-pdf, see
section 3.)  TeX's questions, after an error or from \typein, appear
on your terminal and take your answers, and Ctrl-C reaches TeX, which
answers with its "Interruption" prompt; this holds for resumed builds too.

On a resumed build, AnT first prints what TeX printed up to the snapshot
(saved with it), then the live output, so the terminal transcript is the
same as a plain run's with the same options and -recorder.  So is the
.log file, except for two memory-statistics lines at the very end (AnT
defines one extra macro).  Errors, warnings and SyncTeX refer to your
real file names and line numbers.


2. Installation
---------------

AnT itself needs only Python 3.9 or newer.  It installs and runs without
TeX, but to turn a document into a PDF it calls a TeX engine (lualatex,
pdflatex, xelatex, ...), so for that you need a TeX distribution such as
TeX Live, with LaTeX 2020-10 or newer.

For snapshots, AnT also needs:

  * GNU/Linux (AnT uses LD_PRELOAD and /proc; it is not tested
    elsewhere);
  * a C compiler (cc, gcc or clang) for the snapshot library, which AnT
    compiles from src/ant/antsnap.c on first use and caches in
    ~/.cache/ant/ (in /tmp/ant-<uid>/ if that path contains a space or a
    colon, which LD_PRELOAD cannot handle);
  * an x86-64 machine.

Without a C compiler or on another processor, AnT builds with plain runs
(section 6).

No root access is needed:

    ./install.sh              installs ~/.local/bin/ant and the man page
    ./install.sh --system     installs into /usr/local (uses sudo)
    ./install.sh --uninstall  removes AnT again

The `ant' command is a single executable file (a Python zip application).
If ~/.local/bin is not on your PATH, the installer says what to add.


3. How it works
---------------

The core idea of process checkpointing is to capture a running process and
later continue from that exact point.  Tools such as CRIU do this from
outside the process, which needs root or special privileges.  AnT lets the
engine copy itself instead, with fork(), which any program may do.

  * A small library, src/ant/antsnap.c, is loaded into the unmodified
    engine with LD_PRELOAD.  The engine is started with your command, as
    in section 1 (`lualatex -recorder talk').

  * Pause points.  When the engine opens your main file, the library hands
    it the same text with two hooks added at the start of line 1, so line
    numbers do not change.  TeX still prints and records your file's real
    name.  The hooks make TeX open a special file just before
    \begin{document} and before every \include that TeX actually runs
    (not those skipped by \includeonly or a false conditional).  When the
    library sees that file opened, it notes the position of every open
    file and forks:

      - the copy freezes and becomes a snapshot.  It waits for commands
        and shows up in ps as `ant-snap-<n>'.  It marks itself as the first
        process the kernel's out-of-memory killer should stop, so if
        memory runs out, the kernel ends snapshots before your other
        programs (other memory managers, such as systemd-oomd or
        earlyoom, may choose differently);

      - the original reports to AnT, waits while AnT saves copies of the
        output files (.pdf, .aux, .log, ...), and carries on typesetting.

  * Resuming.  On the next run, AnT puts the output files back as they
    were at the snapshot and tells that frozen copy to fork again.  The
    new copy restores every file position and continues typesetting from
    the pause point; for the main file it also discards text that had been
    read ahead, so an edited body is read fresh.  It reads its terminal
    input from the `ant' command that resumed it and gets back the
    engine's own signal handlers, so TeX can ask questions and Ctrl-C
    works as without AnT.  The frozen copy stays frozen (holding no
    terminal) and can be resumed again and again.

  * Memory.  A snapshot holds what the engine holds at that point, mostly
    fonts and packages.  fork() is copy-on-write, so further snapshots of
    the same run only cost the memory pages changed in between: all 13
    snapshots of the 220-page test book took 330 to 400MB in our tests.
    `ant -v' shows the figure for your document.

  * Lifetime.  Snapshots are ordinary processes, so they survive between
    runs of AnT and the closing of the terminal.  They end:

      - at a reboot, like every process;
      - at a logout only if the system ends a user's processes then (with
        systemd, when KillUserProcesses is on in logind.conf; distributions
        differ);
      - after 8 hours without use (ANT_TTL), with `ant --clean', or within
        half a minute after AnT's cache folder for the document is deleted;
      - if the kernel runs out of memory (see above).

    AnT does not watch for any of this.  At the next build it finds that
    the snapshots are gone, and restores the preamble snapshot from disk
    (section 4) or makes a full run.

  * Which engine.  AnT finds it from the program your command finally
    runs: pdflatex and pdflatex-dev run pdfTeX, lualatex runs LuaHBTeX,
    xelatex runs XeTeX.  Any command that runs pdfTeX, LuaTeX (also
    LuaHBTeX, LuaJITTeX) or XeTeX works.

  * Which file.  TeX's first line is what follows the options: usually the
    file name, as in `lualatex thesis'.  It may also be TeX code, as in
    `pdflatex '\nonstopmode\input paper.ltx''; AnT then takes the file of
    the first \input as the main file.  Without an \input, AnT runs the
    command as it is, without snapshots, and says so.

  * Where AnT keeps its data.  Everything goes to ~/.cache/ant/
    ($XDG_CACHE_HOME/ant/ if that is set), one folder per document,
    readable only by you.  Nothing is written next to the document except
    TeX's own output (including <job>.fls), so a folder synchronised
    between computers (Nextcloud, Syncthing, ...) carries no snapshots;
    each computer keeps its own.  Test versions before 0.1.0 kept their
    data in a .ant/ folder next to the document; AnT stops the snapshots
    left there and removes this document's files from it.

  * Draft and final builds.  LaTeX reads the previous run's .aux (and
    .toc, ...) at \begin{document} and writes new ones during the run.  A
    normal build always does one pass, so a new or moved label shows as
    ?? or with an old number until you run `ant --final'.  That reruns
    from the preamble snapshot (taken before the .aux is read) until the
    .aux, .toc and similar files stop changing, like latexmk (at most 4
    passes).  In drafts, references keep the values read by the last
    build that started at \begin{document} (a full run, a final build, or
    an edit that resumed from the preamble snapshot); a draft that resumes
    from a chapter snapshot uses what that snapshot read.  The text and
    page layout are always current.

  * Change tracking.  Every file the engine reads is recorded (-recorder)
    and assigned to the part of the run that first read it: the preamble,
    the main file's body, or the stretch after one of the pauses.  The
    parts follow what TeX actually did, so \includeonly, conditionals and
    \include inside an \input file are handled correctly.  A change to a
    file read in the preamble needs a full run; any change to the main
    file's body resumes from the preamble snapshot; any other change
    resumes from the snapshot just before the part that first read the
    file.  The library also records every file TeX
    looked for but did not find in the document's folder, so a new
    chapter, image, .bbl (after biber) or .ind (after makeindex) is
    noticed too.  Files that other programs write for TeX are also
    watched by name: .bbl, .ind, .gls, .acr, .nls, .glstex, and glossary
    types declared in the .aux.  More can be added with ANT_WATCH.

  * Errors.  Snapshots taken before a LaTeX error stay valid, so after
    you fix the error AnT resumes just before the chapter that failed.

  * XeTeX runs with -no-pdf, so snapshots never involve its separate PDF
    converter (a second process, which a snapshot cannot hold); AnT runs
    xdvipdfmx once at the end.

  * SyncTeX.  TeX writes <job>.synctex(busy) during the run and renames
    it to <job>.synctex.gz at the end.  A snapshot holds the busy file
    open, so the snapshot library keeps that file in AnT's cache, and the
    renaming becomes a copy to <job>.synctex.gz.  So snapshots work with
    SyncTeX on, and the .synctex.gz file is the same as a plain run's.


4. Snapshots on disk
--------------------

After a build that made a new preamble snapshot, a copy of that snapshot
writes itself to AnT's cache (section 3) in the background, unless it
would be larger than ANT_SAVE_LIMIT, so it survives a reboot.  Only what
is needed to rebuild it is stored:

  * memory backed by files (the engine, its libraries, mapped fonts) is
    mapped again from the files; only pages that differ are stored;
  * empty pages are skipped;
  * the rest is compressed with zstd (zlib if zstd is not installed).

Saving happens in a copy of the frozen engine, which reads the memory of a
pristine child of its own (a parent may read its child's memory without
root), so no build waits for it.

Restoring needs no root either.  AnT starts the engine exactly as it
started the original: same command, recorded environment, and
address-space randomisation switched off for the engine (allowed for any
user), so the binary, libraries, stack and vDSO land at the same
addresses.  Before the engine's main() runs, the snapshot library
replaces the process's memory with the image, reopens its files at their
positions, restores signal handlers, working directory and thread pointer,
and continues from the frozen point.

An image is used only on the same kernel, with the same engine binary,
libraries and mapped files; otherwise AnT makes a full run and says why.
ANT_SAVE_LIMIT (in MB, default 20) sets the largest image kept; 0 turns
saving off.  Images contain the engine's memory, including its environment
variables, which is why AnT's cache is readable only by you.

Measured as an ordinary user:

    document                     engine memory  on disk  restore
    article, pdfLaTeX                     28MB    6.7MB     0.1s
    article, XeLaTeX                     113MB    8.4MB     0.2s
    article, LuaLaTeX                     96MB   13.5MB     0.2s
    220-page book, LuaLaTeX               90MB   13.3MB     0.2s
    talk with a Japanese font            719MB   55.5MB     1.1s
                                                 (over the default limit)


5. Measured results
-------------------

Test files are in testbook/ (run `python3 make_chapters.py' there first).
Machine: TeX Live 2023, LuaLaTeX, as an ordinary user.

book.tex: 220 pages, 12 \include'd chapters with TikZ, pgfplots, siunitx,
hyperref and cleveref.  A plain lualatex run takes 8.5s.  Medians of three
draft builds after an edit:

    edit in            AnT     compared with a plain run
    chapter 12         0.77s   11 times as fast
    chapter 11         1.45s   5.8 times as fast
    chapter 9          2.63s   3.2 times as fast
    chapter 6          4.54s   1.9 times as fast
    chapter 1          7.54s   1.1 times as fast
    preamble           9.09s   7% slower (a full run)
    nothing changed    0.08s   build skipped

article.tex (the same preamble, one chapter, 20 pages): 2.07s plain,
0.76s with AnT after an edit (63% less time).

Correctness: after `ant --final', the book's PDF was compared with plain
lualatex run on the same sources; all 220 pages were identical, also after
an edit that changes references across chapters.  The same held for
pdfLaTeX and XeLaTeX, and for builds restored from disk.


6. Building without snapshots
-----------------------------

When snapshots are not possible (no C compiler, not an x86-64 machine), or
with --no-snapshots, AnT builds with plain runs:

  * pdfTeX, and XeTeX when the preamble loads no system fonts: the
    preamble is cached once in a private format (mylatexformat) and
    reused until it changes, unless TeX's first line is TeX code;
  * LuaTeX, and XeTeX with system fonts: these fonts cannot be stored in
    a format file, so the preamble is processed normally;
  * all engines: reruns until references settle with --final, and no run
    at all when no input changed.


7. Options and environment
--------------------------

    ant [AnT options] TeX-command [TeX options] file
    ant [AnT options] -q file

AnT's options come first; the first word that is not one of them starts
the TeX command (`--' ends AnT's options explicitly).

    -q, --quick          the file only, with the default engine:
                         `ant -q thesis' is `ant lualatex thesis'
    -F, --final          rerun until references, citations and table of
                         contents are up to date
    -d, --draft          one pass; references not updated (the default)
    -v, --verbose        also show passes, changed files, snapshots and
                         their memory
    --silent             show only TeX's output and AnT's error messages
    --no-snapshots       build with plain runs instead of snapshots
    -f, --force          stop snapshots, discard cached state, build from
                         scratch
    --clean              stop this document's snapshots, delete its cache,
                         and exit (`ant --clean lualatex thesis')
    -V, --version        show version, copyright and license
    -h, --help           show a short help text

TeX's options go to TeX unchanged (-shell-escape, -synctex=1,
-interaction=batchmode, -file-line-error, -jobname=NAME, ...).  AnT does
not support these yet, and says so: -ini, -fmt (and &format),
-output-directory, -aux-directory, -output-format and -output-driver.  A
change of the TeX command makes the next build a full run.

    ANT_ENGINE           the command for -q instead of lualatex
    ANT_SILENT           if set, as --silent
    ANT_TTL              seconds an unused snapshot lives in memory
                         (default 28800, i.e. 8 hours)
    ANT_SAVE_LIMIT       largest preamble snapshot kept on disk, in MB
                         (default 20; 0 = never)
    ANT_WATCH            more generated-file extensions to watch
    ANT_SHIM             use this snapshot library instead of building one
    ANT_DEBUG            if set, explain why a snapshot could not be used


8. Limitations
--------------

  * Snapshots use memory while they live, roughly what the engine itself
    needs for your fonts and packages.  Use `ant --clean' when done, or a
    lower ANT_TTL.
  * Chapter snapshots happen only at \include; documents that use only
    \input get the preamble snapshot.  With \includeonly, only the
    chapters it includes get snapshots.
  * The TeX options -ini, -fmt, -output-directory, -aux-directory,
    -output-format and -output-driver are not supported yet.
  * Snapshots keep the output files (.pdf, .aux, ...) open.  If another
    program replaces one of them (some synchronisation tools do), the
    snapshots that hold it cannot be used, and AnT starts from an earlier
    one or makes a full run.
  * Only the preamble snapshot is saved on disk; after a reboot the first
    build resumes from the preamble and makes the chapter snapshots again.
  * Disk images need the same kernel and TeX installation as when they were
    saved (checked; otherwise a full run).  Where processes may not read
    their children's memory (Yama ptrace scope 2 or 3), images cannot be
    saved; snapshots in memory still work.
  * The engine runs with address-space randomisation switched off, which
    restoring images requires.  This affects only the engine AnT starts.
  * AnT's hooks are added to the start of line 1 of the copy of the main
    file that the engine reads, so a %&format line or a byte-order mark at
    the very start of the file is not recognised.
  * The pause uses read stream 15 (\openin15) for an instant; it would
    clash only with a document that keeps stream 15 open across
    \begin{document} or an \include.
  * \today, time stamps and \write18 side effects come from the run that
    made the snapshot.
  * The engine must be dynamically linked (true for TeX Live and the
    usual distribution packages), because the library is loaded with
    LD_PRELOAD.


9. Files in this package
------------------------

    README                 this file
    COPYING                the GNU General Public License, version 3
    AUTHORS                who wrote AnT
    NEWS                   changes between versions
    MESSAGES               every message, the help page and the man page
    ant.1                  the man page
    install.sh             builds the single-file `ant' and installs it
    pyproject.toml         alternative installation with pip or pipx
    src/ant/cli.py         command line
    src/ant/messages.py    every text AnT prints
    src/ant/snapshot.py    builds with snapshots
    src/ant/antsnap.c      the snapshot library (LD_PRELOAD)
    src/ant/shim.py        compiles and locates the snapshot library
    src/ant/fallback.py    builds without snapshots
    src/ant/common.py      shared helpers
    tools/catalogue.py     regenerates MESSAGES from the source
    testbook/              test documents (book.tex, article.tex,
                           talk.tex, make_chapters.py)


10. Copyright and license
-------------------------

Copyright © 2026 निरंजन (Niranjan)

AnT is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation, either version 3 of the License, or (at your
option) any later version.

AnT is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
for more details.

You should have received a copy of the GNU General Public License
along with AnT.  If not, see <https://www.gnu.org/licenses/>.

AnT was written with the help of Claude Opus 5.5, a large language model
(LLM) made by Anthropic.  Courts are divided on whether material
generated by an LLM can be protected by copyright.  Should it be found
that the copyright claimant cannot hold copyright in any part of AnT,
that part is in the public domain.  Wherever copyright does hold, AnT is
distributed under the terms of the GNU General Public License, version 3
or (at your option) any later version, as stated above.