Github Assets Cnb

2021-04-20 712 words 2 minutes

Contents

在構建 container image 時，有時候會有需求動態下載 Github repo 中的 Assets 檔案，簡單的方式就是在 dockerfile 透過 curl 指令來獲取檔案，如果是 private repo 時另外配置 TOKEN 即可

dockerfile

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
ARG GITHUB_TOKEN=<github-token>
ARG REPO=qeek-dev/apitest-toolchain
ARG FILE=apitest-toolchain-linux-amd64
ARG VERSION=v0.1.0
ARG TARGET=apitest-toolchain

RUN ASSET_ID=$(if [ $VERSION != "latest" ]; then \
        echo $(curl -H "Authorization: token $GITHUB_TOKEN" -H "Accept: application/vnd.github.v3.raw" -s https://api.github.com/repos/$REPO/releases | jq ". | map(select(.VERSION == \"$VERSION\"))[0].assets | map(select(.name == \"$FILE\"))[0].id"); \
    else\
        echo $(curl -H "Authorization: token $GITHUB_TOKEN" -H "Accept: application/vnd.github.v3.raw" -s https://api.github.com/repos/$REPO/releases | jq ".[0].assets | map(select(.name == \"$FILE\"))[0].id"); \
    fi) \
    && curl -vLJo /tmp/$TARGET -H 'Accept: application/octet-stream' "https://$GITHUB_TOKEN:@api.github.com/repos/$REPO/releases/assets/$ASSET_ID" \
    && chmod +x /tmp/$TARGET && mv /tmp/$TARGET /bin

上述 dockerfile 可以分成二個部份

透過 Github API 獲取對應 Asset ID
透過 curl 下載對應 Asset ID 的實體檔案

也因為自己有這一個需求，所以 cage1016/github-assets-cnb 這一個 Buildapck 的專案就出現了

cage1016/github-assets-cnb

A Cloud Native Buildpack that Download Github Assets

基本上這一個 buildpack 就是透過 paketo-buildpacks/packit 這一個 Buildpacks Utils Library 來實作，一定程度減輕了對 buildpacks/spec 有點麻煩的規範

例如

github.com/paketo-buildpacks/packit/cargo: 有提供對 metadata.dependencies 操作的函式
github.com/paketo-buildpacks/packit/postal: 有提供 Drop 可以直接透過 http 來下載檔案

Usage

Create a folder
1
mkdir -p sample-app

create project.toml, 這邊以下載 skaffold 為例，如果 Github Asset 是屬於 private repo，配置加上 TOKEN 即可

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
cat <<EOF >> sample-app/project.toml
# [[build.env]]
# optional, github token for private assets
# name = "TOKEN"
# value = "<github-token>"

# skaffold
[[build.env]]
# required
name = "REPO"
value = "GoogleContainerTools/skaffold"

[[build.env]]
# required
name = "FILE"
value = "skaffold-linux-amd64"

[[build.env]]
# optional, default set to FILE value
name = "TARGET"
value = "skaffold"

[[build.env]]
# optional, default set to 'latest'
name = "VERSION"
value = "v1.22.0"
EOF

build container image with buildpack

1
2
3
4
pack build test_assets_run --path ./sample-app \
    -b cage1016/[email protected] \
    --builder gcr.io/buildpacks/builder:v1 \
    && docker run --rm test_assets_run "skaffold version"

demo
詳細實作的細節請至 cage1016/github-assets-cnb 查閱

[email protected]

[email protected] 我們在 project.toml 中配置相關的的 [[build.env]] 來指定對應的 Github Assets 參數。基本上可以達成一開始的期望在 Buildpack 構建 Container image 過期程中下載 Github Assses 的檔案，不過如果有需求下載多個 Assets 時就沒有辦法滿足這個需求了

所以在 [email protected] 中增加了

Support Download Public/Private Github Assets
x-tar, gzip, x-zx, zip auto unarchive

Support metadata.githubassets fields

repo: Github Repo
asset: Github Repo asset name
tag: Release tag name, default set to “latest”
token_env: (optional), Please assign ENV name for private repo
destination: download asset destination path to, bin/<your-asset> for application/x-executable asset
strip_components: x-tar, gzip, x-zx suuport StripComponents feature.

Create project.toml if you want to embed github assets

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
cat <<EOF >> project.toml
# assign token
[[build.env]]
name = "APITEST_TOOLCHAIN_TOKEN"
value = "<github-token>"

[[metadata.githubassets]]
repo = "kkdai/youtube"
asset = "youtubedr_2.7.0_linux_arm64.tar.gz"
destination = "bin"

[[metadata.githubassets]]
repo = "qeek-dev/apitest-toolchain"
token_env = "APITEST_TOOLCHAIN_TOKEN"
asset = "apitest-toolchain-linux-amd64"
destination = "bin/apitest-toolchain"
tag = "v0.1.0"

[[metadata.githubassets]]
repo = "stedolan/jq"
asset = "jq-linux64"
destination = "bin/jq"
EOF

Build container image

`1`	`pack build myapp --buildpack cage1016/[email protected]`

Check /layers/cage1016_github-assets-cnb
Container image