sij/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2024-11-23 15:38:55 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 5
khoj/.github/workflows/pypi.yml

81 lines
2 KiB
YAML
Raw Permalink Normal View History

Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
name: pypi
Create Workflow to Publish Application to (test) PyPI
2022-08-03 20:57:27 +02:00
on:
Fix publish of pip package to PyPi on git tag
2022-08-15 05:47:36 +02:00
push:
tags:
Update the publish workflow to also run on any tag push
2023-01-05 00:46:48 +01:00
- "*"
Create Workflow to Publish Application to (test) PyPI
2022-08-03 20:57:27 +02:00
branches:
- 'master'
paths:
Use the src/ layout to fix packaging Khoj for PyPi - Why The khoj pypi packages should be installed in `khoj' directory. Previously it was being installed into `src' directory, which is a generic top level directory name that is discouraged from being used - Changes - move src/* to src/khoj/* - update `setup.py' to `find_packages' in `src' instead of project root - rename imports to form `from khoj.*' in complete project - update `constants.web_directory' path to use `khoj' directory - rename root logger to `khoj' in `main.py' - fix image_search tests to use the newly rename `khoj' logger - update config, docs, workflows to reference new path `src/khoj'
2023-02-14 21:50:51 +01:00
- src/khoj/**
Add developer support for using next.js to serve generated static files (#814) To improve the developer experience for front-end development, we're migrating to Next.js. In order to do this migration page-by-page, we're using static site generation via Next.js. This also helps us avoid making cross site requests from front-end to back-end for the time being, while giving a ramp to separating out server and client if needed for scale down the road. Dev instructions for using the next.js setup are in the added README. This adds scaffolding for including the built files in the python package as well as the docker images. Docker setup has been tested locally. In order to verify the build is working as expected, we can navigate to the {khoj_host}:42110/experimental and verify that the experiment page comes up. This setup works with serving static files included in the src/interface/web folder from the Django app. The key bit for understanding the setup is in the yarn export command in package.json.
2024-06-22 16:42:41 +02:00
- src/interface/web/**
Fix authors, homepage URL in pyproject.toml and workflow triggers
2023-02-16 10:12:04 +01:00
- pyproject.toml
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
- .github/workflows/pypi.yml
Enable the publish workflow for PRs created in the main repo The publish workflow was previously disabled for PRs in commit d1945c5ba8537811e49f1fd5c5b5793e83f0d04d
2023-02-14 20:47:25 +01:00
pull_request:
branches:
- 'master'
paths:
Use the src/ layout to fix packaging Khoj for PyPi - Why The khoj pypi packages should be installed in `khoj' directory. Previously it was being installed into `src' directory, which is a generic top level directory name that is discouraged from being used - Changes - move src/* to src/khoj/* - update `setup.py' to `find_packages' in `src' instead of project root - rename imports to form `from khoj.*' in complete project - update `constants.web_directory' path to use `khoj' directory - rename root logger to `khoj' in `main.py' - fix image_search tests to use the newly rename `khoj' logger - update config, docs, workflows to reference new path `src/khoj'
2023-02-14 21:50:51 +01:00
- src/khoj/**
Add developer support for using next.js to serve generated static files (#814) To improve the developer experience for front-end development, we're migrating to Next.js. In order to do this migration page-by-page, we're using static site generation via Next.js. This also helps us avoid making cross site requests from front-end to back-end for the time being, while giving a ramp to separating out server and client if needed for scale down the road. Dev instructions for using the next.js setup are in the added README. This adds scaffolding for including the built files in the python package as well as the docker images. Docker setup has been tested locally. In order to verify the build is working as expected, we can navigate to the {khoj_host}:42110/experimental and verify that the experiment page comes up. This setup works with serving static files included in the src/interface/web folder from the Django app. The key bit for understanding the setup is in the yarn export command in package.json.
2024-06-22 16:42:41 +02:00
- src/interface/web/**
Fix authors, homepage URL in pyproject.toml and workflow triggers
2023-02-16 10:12:04 +01:00
- pyproject.toml
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
- .github/workflows/pypi.yml
Support workflow dispatch events for running the pypi.yml job
2024-08-06 12:25:39 +02:00
workflow_dispatch:
Create Workflow to Publish Application to (test) PyPI
2022-08-03 20:57:27 +02:00
jobs:
Publish to Test PyPI on Push to PR or Master Branch
2022-08-04 02:49:32 +02:00
publish:
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
name: Publish Python Package to PyPI
Update stale Khoj pypi package metadata Use latest License, Intended Audience and Dev Status
2024-03-28 19:28:15 +01:00
runs-on: ubuntu-latest
Use OIDC TrustedPublisher to publish khoj python package to PyPi
2024-03-28 18:28:36 +01:00
permissions:
id-token: write
Create Workflow to Publish Application to (test) PyPI
2022-08-03 20:57:27 +02:00
steps:
Migrate the pypi package to khoj project name. Update references - Deprecate khoj-assistant pypi package. Use more accurate and succinct pypi project name, khoj - Update references to sye khoj pypi package in docs and code instead of the legacy khoj-assistant pypi package - Update pypi workflow to publish to both khoj, khoj-assistant for now - Update stale python 3.9 support mentioned in our pyproject. Can't support python 3.9 as depend on latest django which support >=3.10
2024-07-16 18:15:41 +02:00
- uses: actions/checkout@v4
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
with:
fetch-depth: 0
Create Workflow to Publish Application to (test) PyPI
2022-08-03 20:57:27 +02:00
Increase minimum python required in the pyproject, use python 3.11 for building the wheel in the workflow
2024-03-28 07:49:07 +01:00
- name: Set up Python 3.11
Create Workflow to Publish Application to (test) PyPI
2022-08-03 20:57:27 +02:00
uses: actions/setup-python@v4
with:
Increase minimum python required in the pyproject, use python 3.11 for building the wheel in the workflow
2024-03-28 07:49:07 +01:00
python-version: '3.11'
Create Workflow to Publish Application to (test) PyPI
2022-08-03 20:57:27 +02:00
Migrate the pypi package to khoj project name. Update references - Deprecate khoj-assistant pypi package. Use more accurate and succinct pypi project name, khoj - Update references to sye khoj pypi package in docs and code instead of the legacy khoj-assistant pypi package - Update pypi workflow to publish to both khoj, khoj-assistant for now - Update stale python 3.9 support mentioned in our pyproject. Can't support python 3.9 as depend on latest django which support >=3.10
2024-07-16 18:15:41 +02:00
- name: ⬇️ Install Server
Fix workflow to publish Khoj python package to PyPi
2023-02-15 04:44:00 +01:00
run: python -m pip install --upgrade pip && pip install --upgrade .
Create Workflow to Publish Application to (test) PyPI
2022-08-03 20:57:27 +02:00
Migrate the pypi package to khoj project name. Update references - Deprecate khoj-assistant pypi package. Use more accurate and succinct pypi project name, khoj - Update references to sye khoj pypi package in docs and code instead of the legacy khoj-assistant pypi package - Update pypi workflow to publish to both khoj, khoj-assistant for now - Update stale python 3.9 support mentioned in our pyproject. Can't support python 3.9 as depend on latest django which support >=3.10
2024-07-16 18:15:41 +02:00
- name: ⬇️ Install Web Client
Add developer support for using next.js to serve generated static files (#814) To improve the developer experience for front-end development, we're migrating to Next.js. In order to do this migration page-by-page, we're using static site generation via Next.js. This also helps us avoid making cross site requests from front-end to back-end for the time being, while giving a ramp to separating out server and client if needed for scale down the road. Dev instructions for using the next.js setup are in the added README. This adds scaffolding for including the built files in the python package as well as the docker images. Docker setup has been tested locally. In order to verify the build is working as expected, we can navigate to the {khoj_host}:42110/experimental and verify that the experiment page comes up. This setup works with serving static files included in the src/interface/web folder from the Django app. The key bit for understanding the setup is in the yarn export command in package.json.
2024-06-22 16:42:41 +02:00
run: |
yarn install
Change the way the export is created for the pypi package in order to transfer static files out of the tmp shell
2024-08-05 19:16:54 +02:00
yarn pypiciexport
Add developer support for using next.js to serve generated static files (#814) To improve the developer experience for front-end development, we're migrating to Next.js. In order to do this migration page-by-page, we're using static site generation via Next.js. This also helps us avoid making cross site requests from front-end to back-end for the time being, while giving a ramp to separating out server and client if needed for scale down the road. Dev instructions for using the next.js setup are in the added README. This adds scaffolding for including the built files in the python package as well as the docker images. Docker setup has been tested locally. In order to verify the build is working as expected, we can navigate to the {khoj_host}:42110/experimental and verify that the experiment page comes up. This setup works with serving static files included in the src/interface/web folder from the Django app. The key bit for understanding the setup is in the yarn export command in package.json.
2024-06-22 16:42:41 +02:00
working-directory: src/interface/web
Try to manually copy the built files into where the src directory should be for the pypi build
2024-08-06 06:35:06 +02:00
- name: 📂 Copy Generated Files
run: |
Fix directory referenceds in pypi.yml configuration for compiled folder
2024-08-06 10:01:42 +02:00
mkdir -p src/khoj/interface/compiled
Fix file paths used for pypi wheel building
2024-09-22 21:42:08 +02:00
cp -r /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/khoj/interface/compiled/* src/khoj/interface/compiled/
Try to manually copy the built files into where the src directory should be for the pypi build
2024-08-06 06:35:06 +02:00
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
- name: ⚙️ Build Python Package
run: |
Fix setup of Reproducible Build environment in publish workflow - Note: Reproducible builds have not been validated. This is just preliminary work to get there. Further testing and fixes maybe required
2022-08-19 20:00:12 +02:00
# Setup Environment for Reproducible Builds
export PYTHONHASHSEED=42
export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
Fix workflow to publish Khoj python package to PyPi
2023-02-15 04:44:00 +01:00
rm -rf dist
Fix setup of Reproducible Build environment in publish workflow - Note: Reproducible builds have not been validated. This is just preliminary work to get there. Further testing and fixes maybe required
2022-08-19 20:00:12 +02:00
Stop pushing deprecated khoj-assistant package to pypi - Also skip uploading package version to it already exists on pypi This happens when a release is new khoj tagged release is created
2024-08-20 20:40:07 +02:00
# Build PyPI Package
Fix workflow to publish Khoj python package to PyPi
2023-02-15 04:44:00 +01:00
pipx run build
Check wheel contents in workflow before publishing it to PyPI
2023-02-14 20:41:33 +01:00
Emojify step names in workflows. Stop publishing to TestPyPi from PR
2023-03-01 17:56:39 +01:00
- name: 🌡️ Validate Python Package
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
run: |
Check wheel contents in workflow before publishing it to PyPI
2023-02-14 20:41:33 +01:00
# Validate PyPi Package
Ignore-check step failed due to unrecognized code. Try using capital letters for indicator
2023-11-21 20:33:43 +01:00
pipx run check-wheel-contents dist/*.whl --ignore W004
Fix workflow to publish Khoj python package to PyPi
2023-02-15 04:44:00 +01:00
pipx run twine check dist/*
Check wheel contents in workflow before publishing it to PyPI
2023-02-14 20:41:33 +01:00
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
- name: ⏫ Upload Python Package Artifacts
Migrate the pypi package to khoj project name. Update references - Deprecate khoj-assistant pypi package. Use more accurate and succinct pypi project name, khoj - Update references to sye khoj pypi package in docs and code instead of the legacy khoj-assistant pypi package - Update pypi workflow to publish to both khoj, khoj-assistant for now - Update stale python 3.9 support mentioned in our pyproject. Can't support python 3.9 as depend on latest django which support >=3.10
2024-07-16 18:15:41 +02:00
uses: actions/upload-artifact@v4
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
with:
Migrate the pypi package to khoj project name. Update references - Deprecate khoj-assistant pypi package. Use more accurate and succinct pypi project name, khoj - Update references to sye khoj pypi package in docs and code instead of the legacy khoj-assistant pypi package - Update pypi workflow to publish to both khoj, khoj-assistant for now - Update stale python 3.9 support mentioned in our pyproject. Can't support python 3.9 as depend on latest django which support >=3.10
2024-07-16 18:15:41 +02:00
name: khoj
path: dist/khoj-*.whl
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
Stop pushing deprecated khoj-assistant package to pypi - Also skip uploading package version to it already exists on pypi This happens when a release is new khoj tagged release is created
2024-08-20 20:40:07 +02:00
- name: 📦 Publish Python Package to PyPI
Improve workflow to publish khoj to pypi - Use emoji's to improve visual indicator of action step - Rename to pypi instead of the more ambiguous publish name Publish could mean publish docker image, publish to pypi, MELPA or Obsidian plugin - Update workflow badge, link pypi badge to khoj pypi package page - Use pypa official github action to upload package to (test) pypi instead of doing it manually using twine - Upload python package artifact for easier access for testing. As uploading to testpypi doesn't work for PRs by others from forked repos
2023-02-15 02:47:42 +01:00
if: startsWith(github.ref, 'refs/tags') || github.ref == 'refs/heads/master'
Specify version of pypi gh publish action
2024-03-28 08:17:31 +01:00
uses: pypa/gh-action-pypi-publish@v1.8.14
Stop pushing deprecated khoj-assistant package to pypi - Also skip uploading package version to it already exists on pypi This happens when a release is new khoj tagged release is created
2024-08-20 20:40:07 +02:00
with:
skip-existing: true
Reference in a new issue Copy permalink