Reduce Debian, Mac App Size. Remove unused libraries under Torch/Lib

## Changes
- On **Debian**
  - `libtorch_cuda.so` (1Gb) and `libtorch_cpu.so` (700Mb) are large shared libs
  - They are available at package root and under `torch/lib` directory in the package
  - We remove the unused, duplicate libraries from under `torch/lib` as only the top level libraries are used
- On **Mac**
  - Remove `libtorch_{cpu,python}.dylib` under `torch/lib` directory from the Mac app.

## App Size Reduction
  - **Debian amd64** app size by **42%** from **1.6Gb to 920Mb**  
  - **Mac arm64** app by **15%** from **190Mb to 160Mb**
  - **Mac amd64** app by **33%** from **340Mb to 230Mb**

## Reference
- [Release Workflow Run](https://github.com/debanjum/khoj/actions/runs/2878104171) after changes
- [Release Workflow Run](https://github.com/debanjum/khoj/actions/runs/2869906116) before changes
This commit is contained in:
Debanjum 2022-08-17 20:48:56 +00:00 committed by GitHub
commit b78ee317ae
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 23 additions and 8 deletions

View file

@ -49,9 +49,9 @@ jobs:
run: |
# Setup Environment for Reproducible Builds
export PYTHONHASHSEED=42
export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct
export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
pyinstaller Khoj.spec
pyinstaller --noconfirm Khoj.spec
if [ "$RUNNER_OS" == "Windows" ]; then
mv dist/Khoj.exe dist/khoj_"$GITHUB_REF_NAME"_amd64.exe
fi

View file

@ -1,4 +1,6 @@
# -*- mode: python ; coding: utf-8 -*-
from os.path import join
from platform import system
from PyInstaller.utils.hooks import copy_metadata
datas = [('src/interface/web', 'src/interface/web')]
@ -29,6 +31,17 @@ a = Analysis(
cipher=block_cipher,
noarchive=False,
)
# Filter out unused, duplicate shared libs under torch/lib
torch_lib_path = set([
join('torch', 'lib', 'libtorch_cuda.so'),
join('torch', 'lib', 'libtorch_cuda.dylib'),
join('torch', 'lib', 'libtorch_cpu.so'),
join('torch', 'lib', 'libtorch_cpu.dylib'),
join('torch', 'lib', 'libtorch_python.so'),
join('torch', 'lib', 'libtorch_python.dylib')])
a.datas = [entry for entry in a.datas if not entry[0] in torch_lib_path]
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)
exe = EXE(
@ -53,9 +66,11 @@ exe = EXE(
entitlements_file=None,
icon='src/interface/web/assets/icons/favicon.icns',
)
app = BUNDLE(
exe,
name='Khoj.app',
icon='src/interface/web/assets/icons/favicon.icns',
bundle_identifier=None,
)
if system() == 'Darwin':
app = BUNDLE(
exe,
name='Khoj.app',
icon='src/interface/web/assets/icons/favicon.icns',
bundle_identifier=None,
)