Compiling (La)TeX files automatically with GitLab CI









up vote
9
down vote

favorite
5












Inspired by this question about compiling LaTeX files with Travis CI:




How can GitLab's integrated CI feature be used to build (La)TeX documents?











share|improve this question























  • Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
    – TeXnician
    Nov 11 at 16:51










  • No, I wouldn’t mind at all
    – JBantje
    Nov 11 at 16:54














up vote
9
down vote

favorite
5












Inspired by this question about compiling LaTeX files with Travis CI:




How can GitLab's integrated CI feature be used to build (La)TeX documents?











share|improve this question























  • Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
    – TeXnician
    Nov 11 at 16:51










  • No, I wouldn’t mind at all
    – JBantje
    Nov 11 at 16:54












up vote
9
down vote

favorite
5









up vote
9
down vote

favorite
5






5





Inspired by this question about compiling LaTeX files with Travis CI:




How can GitLab's integrated CI feature be used to build (La)TeX documents?











share|improve this question















Inspired by this question about compiling LaTeX files with Travis CI:




How can GitLab's integrated CI feature be used to build (La)TeX documents?








compiling automation tools revision-control






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 11 at 16:54









TeXnician

23.8k62984




23.8k62984










asked Nov 11 at 16:33









JBantje

9321512




9321512











  • Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
    – TeXnician
    Nov 11 at 16:51










  • No, I wouldn’t mind at all
    – JBantje
    Nov 11 at 16:54
















  • Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
    – TeXnician
    Nov 11 at 16:51










  • No, I wouldn’t mind at all
    – JBantje
    Nov 11 at 16:54















Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
– TeXnician
Nov 11 at 16:51




Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
– TeXnician
Nov 11 at 16:51












No, I wouldn’t mind at all
– JBantje
Nov 11 at 16:54




No, I wouldn’t mind at all
– JBantje
Nov 11 at 16:54










4 Answers
4






active

oldest

votes

















up vote
7
down vote













Beware: This is a post about ConTeXt



Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).



Basic example compiling a ConTeXt document:



stages:
- build

build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf


More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):



stages:
- build

build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd .. && mkdir fonts && cd fonts
- git clone https://github.com/tonsky/FiraCode firacode
- export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
- mtxrun --script fonts --reload
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf


For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:



FROM alpine:latest

RUN apk update && apk add wget rsync unzip libgcc

RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
sh ./first-setup.sh --modules=all --context=beta --engine=luatex

ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"





share|improve this answer





























    up vote
    4
    down vote













    Backstory



    I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex files and upload them to an enterprise-scale OwnCloud installation.



    I used a Travis CI setup as in the linked question a few years back.



    How does GitLab CI work in general?



    If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …



    This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
    The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).



    For more details, see the documentation of GitLab.



    My Setup for Compiling .tex files



    On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.



    From the .gitlab-ci.yml file a makefile is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.



    Details and my .gitlab-ci.yml file can be found in the repo linked above.



    Pros and Cons of this GitLab CI solution



    (In comparison to the approach with Travis CI and GitHub mentioned earlier)



    Pros:



    • the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation

    • the caching prevents unnecessary compilation

    • a full featured TeXlive installation is used

    • imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)

    • Extremely customisable (up to the TeXlive installation)

    Cons:



    • the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)

    My Verdict



    Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.



    If there are further questions feel free to ask.






    share|improve this answer
















    • 1




      you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
      – David Carlisle
      Nov 11 at 17:24

















    up vote
    2
    down vote













    This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018 image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml file.



    This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.



    image: sumdoc/texlive-2018

    before_script:
    - apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
    - tlmgr update --self --all

    build:
    script:
    - arara -lv document.tex
    artifacts:
    paths:
    - ./*.pdf


    You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:



    image: sumdoc/texlive-2018

    before_script:
    - apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
    - git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
    - fc-cache -fv
    - tlmgr update --self --all

    build:
    script:
    - lualatex document.tex
    artifacts:
    paths:
    - ./*.pdf





    share|improve this answer



























      up vote
      1
      down vote













      For my cv I use the script below:



      compile_pdf:
      image: aergus/latex
      script:
      - latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
      artifacts:
      paths:
      - ./build/resume.pdf


      This is saved in a file called .gitlab-ci.yml and will build a new version every time a new commit is made.



      My .latexmkrc contains the following:



      $latex = 'latex %O --shell-escape %S';
      $pdflatex = 'lualatex %O --shell-escape %S';
      $out_dir = 'build';


      I use the --shell-escape command for the minted package.






      share|improve this answer




















        Your Answer








        StackExchange.ready(function()
        var channelOptions =
        tags: "".split(" "),
        id: "85"
        ;
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function()
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled)
        StackExchange.using("snippets", function()
        createEditor();
        );

        else
        createEditor();

        );

        function createEditor()
        StackExchange.prepareEditor(
        heartbeatType: 'answer',
        convertImagesToLinks: false,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: null,
        bindNavPrevention: true,
        postfix: "",
        imageUploader:
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        ,
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        );



        );













        draft saved

        draft discarded


















        StackExchange.ready(
        function ()
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f459484%2fcompiling-latex-files-automatically-with-gitlab-ci%23new-answer', 'question_page');

        );

        Post as a guest















        Required, but never shown

























        4 Answers
        4






        active

        oldest

        votes








        4 Answers
        4






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes








        up vote
        7
        down vote













        Beware: This is a post about ConTeXt



        Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).



        Basic example compiling a ConTeXt document:



        stages:
        - build

        build:
        stage: build
        image: debian:unstable
        script:
        - apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
        - mkdir context && cd context
        - wget http://minimals.contextgarden.net/setup/first-setup.sh
        - sh ./first-setup.sh --modules=all --context=beta --engine=luatex
        - . tex/setuptex
        - cd ..
        - context document.tex
        artifacts:
        paths:
        - ./*.pdf


        More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):



        stages:
        - build

        build:
        stage: build
        image: debian:unstable
        script:
        - apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
        - mkdir context && cd context
        - wget http://minimals.contextgarden.net/setup/first-setup.sh
        - sh ./first-setup.sh --modules=all --context=beta --engine=luatex
        - . tex/setuptex
        - cd .. && mkdir fonts && cd fonts
        - git clone https://github.com/tonsky/FiraCode firacode
        - export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
        - mtxrun --script fonts --reload
        - cd ..
        - context document.tex
        artifacts:
        paths:
        - ./*.pdf


        For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:



        FROM alpine:latest

        RUN apk update && apk add wget rsync unzip libgcc

        RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
        sh ./first-setup.sh --modules=all --context=beta --engine=luatex

        ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"





        share|improve this answer


























          up vote
          7
          down vote













          Beware: This is a post about ConTeXt



          Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).



          Basic example compiling a ConTeXt document:



          stages:
          - build

          build:
          stage: build
          image: debian:unstable
          script:
          - apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
          - mkdir context && cd context
          - wget http://minimals.contextgarden.net/setup/first-setup.sh
          - sh ./first-setup.sh --modules=all --context=beta --engine=luatex
          - . tex/setuptex
          - cd ..
          - context document.tex
          artifacts:
          paths:
          - ./*.pdf


          More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):



          stages:
          - build

          build:
          stage: build
          image: debian:unstable
          script:
          - apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
          - mkdir context && cd context
          - wget http://minimals.contextgarden.net/setup/first-setup.sh
          - sh ./first-setup.sh --modules=all --context=beta --engine=luatex
          - . tex/setuptex
          - cd .. && mkdir fonts && cd fonts
          - git clone https://github.com/tonsky/FiraCode firacode
          - export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
          - mtxrun --script fonts --reload
          - cd ..
          - context document.tex
          artifacts:
          paths:
          - ./*.pdf


          For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:



          FROM alpine:latest

          RUN apk update && apk add wget rsync unzip libgcc

          RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
          sh ./first-setup.sh --modules=all --context=beta --engine=luatex

          ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"





          share|improve this answer
























            up vote
            7
            down vote










            up vote
            7
            down vote









            Beware: This is a post about ConTeXt



            Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).



            Basic example compiling a ConTeXt document:



            stages:
            - build

            build:
            stage: build
            image: debian:unstable
            script:
            - apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
            - mkdir context && cd context
            - wget http://minimals.contextgarden.net/setup/first-setup.sh
            - sh ./first-setup.sh --modules=all --context=beta --engine=luatex
            - . tex/setuptex
            - cd ..
            - context document.tex
            artifacts:
            paths:
            - ./*.pdf


            More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):



            stages:
            - build

            build:
            stage: build
            image: debian:unstable
            script:
            - apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
            - mkdir context && cd context
            - wget http://minimals.contextgarden.net/setup/first-setup.sh
            - sh ./first-setup.sh --modules=all --context=beta --engine=luatex
            - . tex/setuptex
            - cd .. && mkdir fonts && cd fonts
            - git clone https://github.com/tonsky/FiraCode firacode
            - export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
            - mtxrun --script fonts --reload
            - cd ..
            - context document.tex
            artifacts:
            paths:
            - ./*.pdf


            For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:



            FROM alpine:latest

            RUN apk update && apk add wget rsync unzip libgcc

            RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
            sh ./first-setup.sh --modules=all --context=beta --engine=luatex

            ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"





            share|improve this answer














            Beware: This is a post about ConTeXt



            Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).



            Basic example compiling a ConTeXt document:



            stages:
            - build

            build:
            stage: build
            image: debian:unstable
            script:
            - apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
            - mkdir context && cd context
            - wget http://minimals.contextgarden.net/setup/first-setup.sh
            - sh ./first-setup.sh --modules=all --context=beta --engine=luatex
            - . tex/setuptex
            - cd ..
            - context document.tex
            artifacts:
            paths:
            - ./*.pdf


            More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):



            stages:
            - build

            build:
            stage: build
            image: debian:unstable
            script:
            - apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
            - mkdir context && cd context
            - wget http://minimals.contextgarden.net/setup/first-setup.sh
            - sh ./first-setup.sh --modules=all --context=beta --engine=luatex
            - . tex/setuptex
            - cd .. && mkdir fonts && cd fonts
            - git clone https://github.com/tonsky/FiraCode firacode
            - export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
            - mtxrun --script fonts --reload
            - cd ..
            - context document.tex
            artifacts:
            paths:
            - ./*.pdf


            For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:



            FROM alpine:latest

            RUN apk update && apk add wget rsync unzip libgcc

            RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
            sh ./first-setup.sh --modules=all --context=beta --engine=luatex

            ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"






            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited Nov 11 at 16:56

























            answered Nov 11 at 16:50









            TeXnician

            23.8k62984




            23.8k62984




















                up vote
                4
                down vote













                Backstory



                I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex files and upload them to an enterprise-scale OwnCloud installation.



                I used a Travis CI setup as in the linked question a few years back.



                How does GitLab CI work in general?



                If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …



                This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
                The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).



                For more details, see the documentation of GitLab.



                My Setup for Compiling .tex files



                On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.



                From the .gitlab-ci.yml file a makefile is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.



                Details and my .gitlab-ci.yml file can be found in the repo linked above.



                Pros and Cons of this GitLab CI solution



                (In comparison to the approach with Travis CI and GitHub mentioned earlier)



                Pros:



                • the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation

                • the caching prevents unnecessary compilation

                • a full featured TeXlive installation is used

                • imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)

                • Extremely customisable (up to the TeXlive installation)

                Cons:



                • the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)

                My Verdict



                Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.



                If there are further questions feel free to ask.






                share|improve this answer
















                • 1




                  you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
                  – David Carlisle
                  Nov 11 at 17:24














                up vote
                4
                down vote













                Backstory



                I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex files and upload them to an enterprise-scale OwnCloud installation.



                I used a Travis CI setup as in the linked question a few years back.



                How does GitLab CI work in general?



                If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …



                This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
                The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).



                For more details, see the documentation of GitLab.



                My Setup for Compiling .tex files



                On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.



                From the .gitlab-ci.yml file a makefile is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.



                Details and my .gitlab-ci.yml file can be found in the repo linked above.



                Pros and Cons of this GitLab CI solution



                (In comparison to the approach with Travis CI and GitHub mentioned earlier)



                Pros:



                • the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation

                • the caching prevents unnecessary compilation

                • a full featured TeXlive installation is used

                • imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)

                • Extremely customisable (up to the TeXlive installation)

                Cons:



                • the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)

                My Verdict



                Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.



                If there are further questions feel free to ask.






                share|improve this answer
















                • 1




                  you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
                  – David Carlisle
                  Nov 11 at 17:24












                up vote
                4
                down vote










                up vote
                4
                down vote









                Backstory



                I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex files and upload them to an enterprise-scale OwnCloud installation.



                I used a Travis CI setup as in the linked question a few years back.



                How does GitLab CI work in general?



                If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …



                This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
                The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).



                For more details, see the documentation of GitLab.



                My Setup for Compiling .tex files



                On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.



                From the .gitlab-ci.yml file a makefile is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.



                Details and my .gitlab-ci.yml file can be found in the repo linked above.



                Pros and Cons of this GitLab CI solution



                (In comparison to the approach with Travis CI and GitHub mentioned earlier)



                Pros:



                • the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation

                • the caching prevents unnecessary compilation

                • a full featured TeXlive installation is used

                • imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)

                • Extremely customisable (up to the TeXlive installation)

                Cons:



                • the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)

                My Verdict



                Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.



                If there are further questions feel free to ask.






                share|improve this answer












                Backstory



                I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex files and upload them to an enterprise-scale OwnCloud installation.



                I used a Travis CI setup as in the linked question a few years back.



                How does GitLab CI work in general?



                If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …



                This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
                The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).



                For more details, see the documentation of GitLab.



                My Setup for Compiling .tex files



                On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.



                From the .gitlab-ci.yml file a makefile is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.



                Details and my .gitlab-ci.yml file can be found in the repo linked above.



                Pros and Cons of this GitLab CI solution



                (In comparison to the approach with Travis CI and GitHub mentioned earlier)



                Pros:



                • the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation

                • the caching prevents unnecessary compilation

                • a full featured TeXlive installation is used

                • imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)

                • Extremely customisable (up to the TeXlive installation)

                Cons:



                • the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)

                My Verdict



                Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.



                If there are further questions feel free to ask.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 11 at 16:33









                JBantje

                9321512




                9321512







                • 1




                  you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
                  – David Carlisle
                  Nov 11 at 17:24












                • 1




                  you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
                  – David Carlisle
                  Nov 11 at 17:24







                1




                1




                you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
                – David Carlisle
                Nov 11 at 17:24




                you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
                – David Carlisle
                Nov 11 at 17:24










                up vote
                2
                down vote













                This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018 image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml file.



                This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.



                image: sumdoc/texlive-2018

                before_script:
                - apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
                - tlmgr update --self --all

                build:
                script:
                - arara -lv document.tex
                artifacts:
                paths:
                - ./*.pdf


                You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:



                image: sumdoc/texlive-2018

                before_script:
                - apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
                - git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
                - fc-cache -fv
                - tlmgr update --self --all

                build:
                script:
                - lualatex document.tex
                artifacts:
                paths:
                - ./*.pdf





                share|improve this answer
























                  up vote
                  2
                  down vote













                  This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018 image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml file.



                  This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.



                  image: sumdoc/texlive-2018

                  before_script:
                  - apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
                  - tlmgr update --self --all

                  build:
                  script:
                  - arara -lv document.tex
                  artifacts:
                  paths:
                  - ./*.pdf


                  You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:



                  image: sumdoc/texlive-2018

                  before_script:
                  - apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
                  - git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
                  - fc-cache -fv
                  - tlmgr update --self --all

                  build:
                  script:
                  - lualatex document.tex
                  artifacts:
                  paths:
                  - ./*.pdf





                  share|improve this answer






















                    up vote
                    2
                    down vote










                    up vote
                    2
                    down vote









                    This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018 image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml file.



                    This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.



                    image: sumdoc/texlive-2018

                    before_script:
                    - apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
                    - tlmgr update --self --all

                    build:
                    script:
                    - arara -lv document.tex
                    artifacts:
                    paths:
                    - ./*.pdf


                    You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:



                    image: sumdoc/texlive-2018

                    before_script:
                    - apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
                    - git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
                    - fc-cache -fv
                    - tlmgr update --self --all

                    build:
                    script:
                    - lualatex document.tex
                    artifacts:
                    paths:
                    - ./*.pdf





                    share|improve this answer












                    This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018 image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml file.



                    This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.



                    image: sumdoc/texlive-2018

                    before_script:
                    - apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
                    - tlmgr update --self --all

                    build:
                    script:
                    - arara -lv document.tex
                    artifacts:
                    paths:
                    - ./*.pdf


                    You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:



                    image: sumdoc/texlive-2018

                    before_script:
                    - apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
                    - git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
                    - fc-cache -fv
                    - tlmgr update --self --all

                    build:
                    script:
                    - lualatex document.tex
                    artifacts:
                    paths:
                    - ./*.pdf






                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Nov 11 at 16:42









                    TeXnician

                    23.8k62984




                    23.8k62984




















                        up vote
                        1
                        down vote













                        For my cv I use the script below:



                        compile_pdf:
                        image: aergus/latex
                        script:
                        - latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
                        artifacts:
                        paths:
                        - ./build/resume.pdf


                        This is saved in a file called .gitlab-ci.yml and will build a new version every time a new commit is made.



                        My .latexmkrc contains the following:



                        $latex = 'latex %O --shell-escape %S';
                        $pdflatex = 'lualatex %O --shell-escape %S';
                        $out_dir = 'build';


                        I use the --shell-escape command for the minted package.






                        share|improve this answer
























                          up vote
                          1
                          down vote













                          For my cv I use the script below:



                          compile_pdf:
                          image: aergus/latex
                          script:
                          - latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
                          artifacts:
                          paths:
                          - ./build/resume.pdf


                          This is saved in a file called .gitlab-ci.yml and will build a new version every time a new commit is made.



                          My .latexmkrc contains the following:



                          $latex = 'latex %O --shell-escape %S';
                          $pdflatex = 'lualatex %O --shell-escape %S';
                          $out_dir = 'build';


                          I use the --shell-escape command for the minted package.






                          share|improve this answer






















                            up vote
                            1
                            down vote










                            up vote
                            1
                            down vote









                            For my cv I use the script below:



                            compile_pdf:
                            image: aergus/latex
                            script:
                            - latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
                            artifacts:
                            paths:
                            - ./build/resume.pdf


                            This is saved in a file called .gitlab-ci.yml and will build a new version every time a new commit is made.



                            My .latexmkrc contains the following:



                            $latex = 'latex %O --shell-escape %S';
                            $pdflatex = 'lualatex %O --shell-escape %S';
                            $out_dir = 'build';


                            I use the --shell-escape command for the minted package.






                            share|improve this answer












                            For my cv I use the script below:



                            compile_pdf:
                            image: aergus/latex
                            script:
                            - latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
                            artifacts:
                            paths:
                            - ./build/resume.pdf


                            This is saved in a file called .gitlab-ci.yml and will build a new version every time a new commit is made.



                            My .latexmkrc contains the following:



                            $latex = 'latex %O --shell-escape %S';
                            $pdflatex = 'lualatex %O --shell-escape %S';
                            $out_dir = 'build';


                            I use the --shell-escape command for the minted package.







                            share|improve this answer












                            share|improve this answer



                            share|improve this answer










                            answered Nov 12 at 6:23









                            Pascal

                            353313




                            353313



























                                draft saved

                                draft discarded
















































                                Thanks for contributing an answer to TeX - LaTeX Stack Exchange!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid


                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.

                                To learn more, see our tips on writing great answers.





                                Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                                Please pay close attention to the following guidance:


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid


                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.

                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function ()
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f459484%2fcompiling-latex-files-automatically-with-gitlab-ci%23new-answer', 'question_page');

                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown







                                這個網誌中的熱門文章

                                How to read a connectionString WITH PROVIDER in .NET Core?

                                In R, how to develop a multiplot heatmap.2 figure showing key labels successfully

                                Museum of Modern and Contemporary Art of Trento and Rovereto