1
\$\begingroup\$

I have a Dockerfile with ubuntu:22.04 as a base image and a manual Python installation(specific version i.e., 3.11.1) layer which takes a long time. How to cache this manual Python installation layer in the docker image in subsequent CI runs in GitHub actions?

Dockerfile:

FROM --platform=linux/amd64 ubuntu:22.04 as base

USER root

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV DEBIAN_FRONTEND noninteractive

COPY ZscalerRootCertificate-2048-SHA256.crt /usr/local/share/ca-certificates/ZscalerRootCertificate-2048-SHA256.crt

RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y software-properties-common ca-certificates && \
    update-ca-certificates

RUN mkdir /python && cd /python && \
        wget https://www.python.org/ftp/python/3.11.1/Python-3.11.1.tgz && \
       tar -zxvf Python-3.11.1.tgz && \
       cd Python-3.11.1 && \
       ls -lhR && \
       ./configure --enable-optimizations && \
       make install && \
       rm -rf /python

# Fail soon than later, if python wasn't installed
RUN python --version

COPY . /app
WORKDIR /app

RUN python -m pip install -U pip

RUN python3.11 -m pip install --no-cache-dir --trusted-host pypi.org --trusted-host files.pythonhosted.org -r requirements/core-requirements.txt

CMD ["gunicorn", "main:app", "--workers", "2", "--worker-class", "uvicorn.workers.UvicornWorker", "--bind", "0.0.0.0:80"]

GitHub workflow,

CI.yml:

name: Docker Image CI

env:
  CONTAINER_NAME: my-use-case

on:
  workflow_dispatch:

concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

permissions:
  id-token: write
  contents: write

jobs:
  integration-tests:
    name: Integration Test
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: View directory files
        run: |
          pwd
          echo "$PATH"

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: "3.11.1"

      - name: Copy Python binaries to Docker context
        run: |
          mkdir -p Python/3.11.1
          cp -r $RUNNER_TOOL_CACHE/Python/3.11.1/* Python/3.11.1

      - name: View directory files
        run: |
          ls -lhR
          echo "$PATH"

      - name: Install dependencies
        run: |
          python --version
          python -m pip install --upgrade pip

      - name: View directory files
        run: |
          ls -lhR

      - name: Build & push docker image
        uses: docker/build-push-action@v2
        with:
          context: .
          push: false
          tags: ${{ env.CONTAINER_NAME }}:${{ github.run_number }}
          file: ./Dockerfile 
\$\endgroup\$
9
  • \$\begingroup\$ This question is new but has something relevant to my old question \$\endgroup\$ Dec 9, 2023 at 16:22
  • 1
    \$\begingroup\$ Why do you need the final two RUN commands after COPY . /app? ("RUN python -m pip install -U pip" and "RUN python3.11 -m pip install ...") \$\endgroup\$
    – Peilonrayz
    Dec 9, 2023 at 18:24
  • \$\begingroup\$ @Peilonrayz Yeah, now I realize that I can copy the requriements.txt file first, then the RUN command, and then the final COPY . /app command. Thanks for your details suggestion on my previous question. \$\endgroup\$ Dec 9, 2023 at 18:36
  • \$\begingroup\$ Do you need to compile Python yourself? I usually use one of the pre-built Debian-based Python images. This still allows you to use apt to install software. hub.docker.com/_/python \$\endgroup\$
    – Nick ODell
    Dec 10, 2023 at 2:25
  • 1
    \$\begingroup\$ @AjinkyaKamat Whilst you're probably right you shouldn't depend on minor versions. Python makes no guarantee the code will work across minor versions. You can just look at the typing module in 3.5 which radically changed the internals numerous times across minor versions. Additionally I've seen minor versions introduce bugs. Version pinning to a specific version (minor included) isn't really bad when Python makes no guarantee about minor version compatibility. \$\endgroup\$
    – Peilonrayz
    Dec 10, 2023 at 17:50

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.