diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..a9325ad
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,48 @@
+# macOS system files
+.DS_Store
+.AppleDouble
+.LSOverride
+Icon
+._*
+
+# Python cache files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Python virtual environments
+venv/
+env/
+.env/
+.venv/
+
+# IDE specific files
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# Logs and databases
+*.log
+*.sqlite
+*.db
+
+# Distribution / packaging
+dist/
+build/
+*.egg-info/
+
+# Temporary files
+log.txt
+*.tmp
+*.bak
+*.swp
+*~
+
+# Operating System temporary files
+*~
+.fuse_hidden*
+.Trash-*
+.nfs*
+
+mlx_models/
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..36adfbe
--- /dev/null
+++ b/README.md
@@ -0,0 +1,385 @@
+# PATH-worthy Scripts π οΈ
+
+A collection of various scripts I use frequently enough to justify keeping them in my system PATH.
+
+I haven't written documentation for all of these scripts. I might in time. Find documentation for some of the highlights below.
+
+## Installation
+
+1. Clone and enter repository:
+
+```bash
+git clone https://sij.ai/sij/pathScripts.git
+cd pathScripts
+```
+
+2. Add to your system PATH:
+
+macOS / ZSH:
+```bash
+echo "export PATH=\"\$PATH:$PWD\"" >> ~/.zshrc
+source ~/.zshrc
+```
+
+Linux / Bash:
+```bash
+echo "export PATH=\"\$PATH:$PWD\"" >> ~/.bashrc
+source ~/.bashrc
+```
+
+3. Make scripts executable:
+
+```bash
+chmod +x *
+```
+
+---
+
+## π `bates` - PDF Bates Number Tool
+
+Extracts and renames PDFs based on Bates numbers.
+
+### Setup
+```bash
+pip3 install pdfplumber
+# For OCR support:
+pip3 install pytesseract pdf2image
+brew install tesseract poppler # macOS
+# or
+sudo apt-get install tesseract-ocr poppler-utils # Debian
+```
+
+### Usage
+```bash
+bates /path/to/folder --prefix "FWS-" --digits 6 --name-prefix "FWS "
+```
+
+### Key Features
+- Extracts Bates numbers from text/scanned PDFs
+- Renames files with number ranges
+- Prepare files for use with my [Bates Source Link](https://sij.ai/sij/DEVONthink/src/branch/main/Bates%20Source%20Link.scpt#) DEVONthink script
+- Preserves original names in Finder comments
+- OCR support for scanned documents
+- Dry-run mode with `--dry-run`
+
+### Options
+- `--prefix`: The Bates number prefix to search for (default: "FWS-")
+- `--digits`: Number of digits after the prefix (default: 6)
+- `--ocr`: Enable OCR for scanned documents
+- `--dry-run`: Test extraction without renaming files
+- `--name-prefix`: Prefix to use when renaming files
+- `--log`: Set logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+
+### Examples
+```bash
+# Test without making changes
+bates /path/to/pdfs --prefix "FWS-" --digits 6 --dry-run
+
+# Rename files with OCR support
+bates /path/to/pdfs --prefix "FWS-" --digits 6 --name-prefix "FWS " --ocr
+```
+
+### Notes
+- Always test with `--dry-run` first
+- Original filenames are preserved in Finder comments (macOS only)
+- OCR is disabled by default to keep things fast
+
+---
+
+## πͺ `camel` - File Renaming Utility
+
+Renames files in the current directory by splitting camelCase, PascalCase, and other compound words into readable, spaced formats.
+
+### Features
+
+- **Smart Splitting**:
+ - Handles camelCase, PascalCase, underscores (`_`), hyphens (`-`), and spaces.
+ - Preserves file extensions.
+ - Splits on capital letters and numbers intelligently.
+- **Word Detection**:
+ - Uses NLTKβs English word corpus and WordNet to identify valid words.
+ - Common words like "and", "the", "of" are always treated as valid.
+- **Automatic Renaming**:
+ - Processes all files in the current directory (ignores hidden files).
+ - Renames files in-place with clear logging.
+
+### Setup
+1. Install dependencies:
+ ```bash
+ pip3 install nltk
+ ```
+2. Download NLTK data:
+ ```bash
+ python3 -m nltk.downloader words wordnet
+ ```
+
+### Usage
+Run the script in the directory containing the files you want to rename:
+```bash
+camel
+```
+
+### Examples
+Before running the script:
+```plaintext
+Anti-OedipusCapitalismandSchizophrenia_ep7.aax
+TheDawnofEverythingANewHistoryofHumanity_ep7.aax
+TheWeirdandtheEerie_ep7.aax
+```
+
+After running the script:
+```plaintext
+Anti Oedipus Capitalism and Schizophrenia ep 7.aax
+The Dawn of Everything A New History of Humanity ep 7.aax
+The Weird and the Eerie ep 7.aax
+```
+
+### Notes
+- Hidden files (starting with `.`) are skipped.
+- If a word isnβt found in the dictionary, itβs left unchanged.
+- File extensions are preserved during renaming.
+
+---
+
+## π¦ `deps` - Unified Python Dependency Manager
+
+A single script that analyzes `import` statements in .py files and installs dependencies using mamba/conda or pip.
+
+### Usage
+```bash
+deps <subcommand> ...
+```
+
+#### Subcommands
+
+1. **`ls`**
+ Analyzes `.py` files for external imports:
+ - Writes PyPI-available packages to `requirements.txt`.
+ - Writes unavailable packages to `missing-packages.txt`.
+
+ **Examples**:
+ ```bash
+ deps ls # Analyze current directory (no recursion)
+ deps ls -r # Recursively analyze current directory
+ deps ls src # Analyze a 'src' folder
+ deps ls -r src # Recursively analyze 'src'
+ ```
+
+2. **`install`**
+ Installs Python packages either by analyzing local imports or from explicit arguments.
+ - **Conda Environment Detection**: If in a conda environment, tries `mamba` (if installed), else `conda`.
+ - **Fallback** to `pip` if conda tool fails or is unavailable.
+ - **`--no-conda`**: Skip conda/mamba entirely and go straight to pip.
+
+ **Examples**:
+ ```bash
+ deps install # Analyze current folder, install discovered packages (no recursion)
+ deps install -r # Same as above but recursive
+ deps install requests # Directly install 'requests'
+ deps install script.py # Analyze and install packages from 'script.py'
+ deps install -R requirements.txt # Install from a requirements file
+ deps install requests --no-conda # Skip conda/mamba, use pip only
+ ```
+
+### How It Works
+- **Scanning Imports**: Locates `import ...` and `from ... import ...` lines in `.py` files, skipping built-in modules.
+- **PyPI Check**: Uses `urllib` to confirm package availability at `pypi.org`.
+- **Requirements & Missing Packages**: If you run `deps ls`, discovered imports go into `requirements.txt` (available) or `missing-packages.txt` (unavailable).
+- **Installation**: For `deps install`:
+ - If no extra arguments, it auto-discovers imports in the current directory (optionally with `-r`) and installs only PyPI-available ones.
+ - If passed packages, `.py` files, or `-R <reqfile>`, it installs those specifically.
+ - By default, tries conda environment tools first (mamba or conda) if in a conda environment, otherwise pip.
+
+### Notes
+- If `mamba` or `conda` is available in your environment, `deps install` will prefer that. Otherwise, it uses pip.
+- You can run `deps ls` repeatedly to keep updating `requirements.txt` and `missing-packages.txt`.
+
+---
+
+## π `linecount` - Line Counting Tool for Text Files
+
+Recursively counts the total lines in all text files within the current directory, with optional filtering by file extensions.
+
+### Usage
+```bash
+linecount [<extension1> <extension2> ...]
+```
+
+### Examples
+```bash
+linecount # Count lines in all non-binary files
+linecount .py .sh # Count lines only in .py and .sh files
+```
+
+### Key Features
+- **Recursive Search**: Processes files in the current directory and all subdirectories.
+- **Binary File Detection**: Automatically skips binary files.
+- **File Extension Filtering**: Optionally count lines in specific file types (case-insensitive).
+- **Quick Stats**: Displays the number of files scanned and total lines.
+
+### Notes
+- If no extensions are provided, all non-binary files are counted.
+- Use absolute or relative paths when running the script in custom environments.
+
+---
+
+## πͺ `murder` - Force-Kill Processes by Name or Port
+
+A utility script to terminate processes by their name or by the port they are listening on:
+- If the argument is **numeric**, the script will terminate all processes listening on the specified port.
+- If the argument is **text**, the script will terminate all processes matching the given name.
+
+### Usage Examples
+```bash
+# Kill all processes listening on port 8080
+sudo murder 8080
+
+# Kill all processes with "node" in their name
+sudo murder node
+```
+
+### Features & Notes
+- Automatically detects whether the input is a **port** or a **process name**.
+- Uses `lsof` to find processes listening on a specified port.
+- Finds processes by name using `ps` and kills them using their process ID (PID).
+- Ignores the `grep` process itself when searching for process names.
+
+### Notes
+- Requires `sudo` privileges.
+- Use with caution, as it forcefully terminates processes.
+
+---
+
+## π `push` & `pull` - Bulk Git Repository Management
+
+Scripts to automate updates and management of multiple Git repositories.
+
+### Setup
+
+1. **Create a Repository List**
+ Add repository paths to `~/.repos.txt`, one per line:
+ ```plaintext
+ ~/sijapi
+ ~/workshop/Nova/Themes/Neonva/neonva.novaextension
+ ~/scripts/pathScripts
+ ~/scripts/Swiftbar
+ ```
+
+ - Use `~` for home directory paths or replace it with absolute paths.
+ - Empty lines and lines starting with `#` are ignored.
+
+2. **Make Scripts Executable**
+ ```bash
+ chmod +x push pull
+ ```
+
+3. **Run the Scripts**
+ ```bash
+ pull # Pulls the latest changes from all repositories
+ push # Pulls, stages, commits, and pushes local changes
+ ```
+
+### Features
+
+#### `pull`
+- Recursively pulls the latest changes from all repositories listed in `~/.repos.txt`.
+- Automatically expands `~` to the home directory.
+- Skips directories that do not exist or are not Git repositories.
+- Uses `git pull --force` to ensure synchronization.
+
+#### `push`
+- Pulls the latest changes from the current branch.
+- Stages and commits all local changes with an auto-generated message: `Auto-update: <timestamp>`.
+- Pushes updates to the current branch.
+- Configures the `origin` remote automatically if missing, using a URL based on the directory name.
+
+### Notes
+- Both scripts assume `~/.repos.txt` is the repository list file. You can update the `REPOS_FILE` variable if needed.
+- Use absolute paths or ensure `~` is correctly expanded to avoid issues.
+- The scripts skip non-existent directories and invalid Git repositories.
+- `push` will attempt to set the `origin` remote automatically if it is missing.
+
+---
+
+## π `vitals` - System and VPN Diagnostics
+
+The `vitals` script provides detailed system diagnostics, VPN status, DNS configuration, and uptime in JSON format. It integrates with tools like AdGuard Home, NextDNS, and Tailscale for network monitoring.
+
+### Usage
+1. **Set up a DNS rewrite rule in AdGuard Home**:
+ - Assign the domain `check.adguard.test` to your Tailscale IP or any custom domain.
+ - Update the `adguard_test_domain` variable in the script if using a different domain.
+
+2. **Run the script**:
+ ```bash
+ vitals
+ ```
+
+ Example output (JSON):
+ ```json
+ {
+ "local_ip": "192.168.1.2",
+ "wan_connected": true,
+ "wan_ip": "185.213.155.74",
+ "has_tailscale": true,
+ "tailscale_ip": "100.100.100.1",
+ "mullvad_exitnode": true,
+ "mullvad_hostname": "de-ber-wg-001.mullvad.ts.net",
+ "nextdns_connected": true,
+ "nextdns_protocol": "DoH",
+ "adguard_connected": true,
+ "uptime": "up 3 days, 2 hours, 15 minutes"
+ }
+ ```
+
+---
+
+## π `vpn` - Tailscale Exit Node Manager
+
+Privacy-focused Tailscale exit node management with automated logging.
+
+### Setup
+```bash
+pip3 install requests
+```
+
+### Usage
+```bash
+vpn <action> [<country>] # Actions: start, stop, new, shh, to, status
+```
+
+### Actions
+- **`start`**: Connect to a suggested exit node if not already connected.
+- **`stop`**: Disconnect from the current exit node.
+- **`new`**: Switch to a new suggested exit node.
+- **`shh`**: Connect to a random exit node in a privacy-friendly country.
+- **`to <country>`**: Connect to a random exit node in a specific country.
+- **`status`**: Display the current exit node, external IP, and connection duration.
+
+### Features
+- **Privacy-Friendly Quick Selection**: Supports random exit nodes from:
+ `Finland`, `Germany`, `Iceland`, `Netherlands`, `Norway`, `Sweden`, `Switzerland`.
+- **Connection Verification**: Ensures exit node and IP via Mullvad API.
+- **Automated Logging**: Tracks all connections, disconnections, and IP changes in `/var/log/vpn_rotation.txt`.
+- **Default Tailscale arguments**:
+ - `--exit-node-allow-lan-access`
+ - `--accept-dns`
+ - `--accept-routes`
+
+### Examples
+```bash
+vpn start # Connect to a suggested node.
+vpn shh # Connect to a random privacy-friendly node.
+vpn to Germany # Connect to a random exit node in Germany.
+vpn status # Show current connection details.
+vpn stop # Disconnect from the exit node.
+```
+
+### Notes
+- Requires active Tailscale configuration and internet access.
+- Logging is handled automatically in `/var/log/vpn_rotation.txt`.
+- Use `sudo` for actions requiring elevated permissions (e.g., `crontab`).
+
+---
+
+_More scripts will be documented as they're updated. Most scripts include `--help` for basic usage information._
diff --git a/aax2mp3 b/aax2mp3
new file mode 100755
index 0000000..1b2f97d
--- /dev/null
+++ b/aax2mp3
@@ -0,0 +1,39 @@
+#!/usr/bin/env python3
+
+import concurrent.futures
+import subprocess
+import glob
+import os
+import multiprocessing
+
+# Different ways to get CPU count
+logical_cores = os.cpu_count() # All cores including hyperthreading
+physical_cores = multiprocessing.cpu_count() # Same as above
+# For more detailed info on Apple Silicon:
+try:
+ # This works on macOS to get performance core count
+ p_cores = len([x for x in os.sched_getaffinity(0) if x < os.cpu_count()//2])
+except AttributeError:
+ p_cores = physical_cores
+
+print(f"System has {logical_cores} logical cores")
+max_workers = max(1, logical_cores - 2) # Leave 2 cores free for system
+
+def convert_file(aax_file):
+ mp3_file = aax_file.replace('.aax', '.mp3')
+ print(f"Converting {aax_file} to {mp3_file}")
+ subprocess.run(['ffmpeg', '-activation_bytes', os.getenv('AUDIBLE_ACTIVATION_BYTES'),
+ '-i', aax_file, mp3_file], check=True)
+
+aax_files = glob.glob('*.aax')
+if not aax_files:
+ print("No .aax files found in current directory")
+ exit(1)
+
+print(f"Found {len(aax_files)} files to convert")
+print(f"Will convert {max_workers} files simultaneously")
+
+with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
+ list(executor.map(convert_file, aax_files))
+
+
diff --git a/bates b/bates
new file mode 100755
index 0000000..0ade2f7
--- /dev/null
+++ b/bates
@@ -0,0 +1,444 @@
+#!/usr/bin/env python3
+
+"""
+Required packages:
+pip3 install pdfplumber pytesseract pdf2image # pdf2image and pytesseract only needed if using --ocr
+
+System dependencies (only if using --ocr):
+brew install tesseract poppler # on macOS
+# or
+sudo apt-get install tesseract-ocr poppler-utils # on Ubuntu/Debian
+"""
+
+import os
+import sys
+import re
+import argparse
+import logging
+from pathlib import Path
+import tempfile
+import subprocess
+import pdfplumber
+
+def check_dependencies(ocr_enabled):
+ try:
+ if ocr_enabled:
+ import pytesseract
+ from pdf2image import convert_from_path
+ except ImportError as e:
+ print(f"Missing dependency: {e}")
+ print("Please install required packages:")
+ if ocr_enabled:
+ print("pip3 install pytesseract pdf2image")
+ sys.exit(1)
+
+
+import os
+import sys
+import re
+import argparse
+import logging
+from pathlib import Path
+import tempfile
+import subprocess
+
+def setup_logging(log_level):
+ """Configure logging with the specified level."""
+ numeric_level = getattr(logging, log_level.upper(), None)
+ if not isinstance(numeric_level, int):
+ raise ValueError(f'Invalid log level: {log_level}')
+
+ logging.basicConfig(
+ level=numeric_level,
+ format='%(asctime)s - %(levelname)s - %(message)s',
+ datefmt='%Y-%m-%d %H:%M:%S'
+ )
+
+def build_regex_pattern(prefix, num_digits):
+ """Build regex pattern based on prefix and number of digits."""
+ # Escape any special regex characters in the prefix
+ escaped_prefix = re.escape(prefix)
+ # Pattern matches the prefix followed by exactly num_digits digits
+ # and ensures no digits or letters follow
+ pattern = f"{escaped_prefix}\\d{{{num_digits}}}(?![\\d\\w])"
+ logging.debug(f"Generated regex pattern: {pattern}")
+ return pattern
+
+def set_finder_comment(file_path, comment):
+ """Set the Finder comment for a file using osascript."""
+ try:
+ # Escape special characters in both the file path and comment
+ escaped_path = str(file_path).replace('"', '\\"').replace("'", "'\\''")
+ escaped_comment = comment.replace('"', '\\"').replace("'", "'\\''")
+
+ script = f'''
+ osascript -e 'tell application "Finder"
+ set commentPath to POSIX file "{escaped_path}" as alias
+ set comment of commentPath to "{escaped_comment}"
+ end tell'
+ '''
+ subprocess.run(script, shell=True, check=True, stderr=subprocess.PIPE)
+ logging.debug(f"Set Finder comment for {file_path} to: {comment}")
+ return True
+ except subprocess.CalledProcessError as e:
+ logging.error(f"Failed to set Finder comment for {file_path}: {e.stderr.decode()}")
+ return False
+ except Exception as e:
+ logging.error(f"Failed to set Finder comment for {file_path}: {e}")
+ return False
+
+def rename_with_bates(file_path, name_prefix, first_num, last_num):
+ """Rename file using Bates numbers and preserve original name in metadata."""
+ try:
+ path = Path(file_path)
+ original_name = path.name
+ new_name = f"{name_prefix}{first_num}β{last_num}{path.suffix}"
+ new_path = path.parent / new_name
+
+ # First try to set the metadata
+ if not set_finder_comment(file_path, original_name):
+ logging.error(f"Skipping rename of {file_path} due to metadata failure")
+ return False
+
+ # Then rename the file
+ path.rename(new_path)
+ logging.info(f"Renamed {original_name} to {new_name}")
+ return True
+ except Exception as e:
+ logging.error(f"Failed to rename {file_path}: {e}")
+ return False
+
+def ocr_page(pdf_path, page_num):
+ """OCR a specific page of a PDF."""
+ filename = Path(pdf_path).name
+ logging.debug(f"[{filename}] Running OCR on page {page_num}")
+ try:
+ # Import OCR-related modules only when needed
+ import pytesseract
+ from pdf2image import convert_from_path
+
+ # Convert specific page to image
+ images = convert_from_path(pdf_path, first_page=page_num+1, last_page=page_num+1)
+ if not images:
+ logging.error(f"[{filename}] Failed to convert page {page_num} to image")
+ return ""
+
+ # OCR the image
+ with tempfile.NamedTemporaryFile(suffix='.png') as tmp:
+ images[0].save(tmp.name, 'PNG')
+ text = pytesseract.image_to_string(tmp.name)
+ logging.debug(f"[{filename}] Page {page_num} OCR result: '{text}'")
+ return text
+ except Exception as e:
+ logging.error(f"[{filename}] OCR failed for page {page_num}: {str(e)}")
+ return ""
+
+def extract_text_from_page_multilayer(page, pdf_path, page_num):
+ """Extract text from different PDF layers."""
+ filename = Path(pdf_path).name
+ # Get page dimensions
+ width = page.width
+ height = page.height
+
+ # Calculate crop box for bottom fifth of page
+ padding = 2
+ y0 = max(0, min(height * 0.8, height - padding))
+ y1 = max(y0 + padding, min(height, height))
+ x0 = padding
+ x1 = max(x0 + padding, min(width - padding, width))
+
+ crop_box = (x0, y0, x1, y1)
+
+ logging.info(f"[{filename}] Page {page_num}: Dimensions {width}x{height}, crop box: ({x0:.2f}, {y0:.2f}, {x1:.2f}, {y1:.2f})")
+
+ texts = []
+
+ # Method 1: Try regular text extraction
+ try:
+ text = page.crop(crop_box).extract_text()
+ if text:
+ logging.info(f"[{filename}] Page {page_num}: Regular extraction found: '{text}'")
+ texts.append(text)
+ except Exception as e:
+ logging.debug(f"[{filename}] Page {page_num}: Regular text extraction failed: {e}")
+
+ # Method 2: Try extracting words individually
+ try:
+ words = page.crop(crop_box).extract_words()
+ if words:
+ text = ' '.join(word['text'] for word in words)
+ logging.info(f"[{filename}] Page {page_num}: Word extraction found: '{text}'")
+ texts.append(text)
+ except Exception as e:
+ logging.debug(f"[{filename}] Page {page_num}: Word extraction failed: {e}")
+
+ # Method 3: Try extracting characters individually
+ try:
+ chars = page.crop(crop_box).chars
+ if chars:
+ text = ''.join(char['text'] for char in chars)
+ logging.info(f"[{filename}] Page {page_num}: Character extraction found: '{text}'")
+ texts.append(text)
+ except Exception as e:
+ logging.debug(f"[{filename}] Page {page_num}: Character extraction failed: {e}")
+
+ # Method 4: Try extracting annotations
+ try:
+ annots = page.annots
+ if annots and isinstance(annots, list): # Fix for the error
+ for annot in annots:
+ if isinstance(annot, dict) and 'contents' in annot:
+ text = annot['contents']
+ if text and not isinstance(text, str):
+ text = str(text)
+ if text and text.lower() != 'none':
+ logging.info(f"[{filename}] Page {page_num}: Annotation found: '{text}'")
+ texts.append(text)
+ except Exception as e:
+ logging.debug(f"[{filename}] Page {page_num}: Annotation extraction failed: {e}")
+
+ # Method 5: Try extracting text in reverse order
+ try:
+ chars = sorted(page.crop(crop_box).chars, key=lambda x: (-x['top'], x['x0']))
+ if chars:
+ text = ''.join(char['text'] for char in chars)
+ logging.info(f"[{filename}] Page {page_num}: Reverse order extraction found: '{text}'")
+ texts.append(text)
+ except Exception as e:
+ logging.debug(f"[{filename}] Page {page_num}: Reverse order extraction failed: {e}")
+
+ # Method 6: Last resort - flatten and OCR the crop box
+ if not texts:
+ try:
+ logging.info(f"[{filename}] Page {page_num}: Attempting flatten and OCR")
+ # Import needed only if we get this far
+ from pdf2image import convert_from_bytes
+ import pytesseract
+
+ # Convert just this page to image
+ with tempfile.NamedTemporaryFile(suffix='.pdf') as tmp_pdf:
+ # Save just this page to a temporary PDF
+ writer = pdfplumber.PDF(page.page_obj)
+ writer.save(tmp_pdf.name)
+
+ # Convert to image
+ images = convert_from_bytes(open(tmp_pdf.name, 'rb').read())
+ if images:
+ # Crop the image to our area of interest
+ img = images[0]
+ img_width, img_height = img.size
+ crop_box_pixels = (
+ int(x0 * img_width / width),
+ int(y0 * img_height / height),
+ int(x1 * img_width / width),
+ int(y1 * img_height / height)
+ )
+ cropped = img.crop(crop_box_pixels)
+
+ # OCR the cropped area
+ text = pytesseract.image_to_string(cropped)
+ if text:
+ logging.info(f"[{filename}] Page {page_num}: Flatten/OCR found: '{text}'")
+ texts.append(text)
+ except Exception as e:
+ logging.debug(f"[{filename}] Page {page_num}: Flatten/OCR failed: {e}")
+
+ return texts
+
+
+def find_bates_number(texts, pattern):
+ """Try to find Bates number in multiple text layers."""
+ for text in texts:
+ matches = list(re.finditer(pattern, text))
+ if matches:
+ return matches[-1] # Return last match if found
+ return None
+
+def extract_bates_numbers(pdf_path, pattern, use_ocr):
+ """Extract Bates numbers from first and last page of PDF using provided pattern."""
+ filename = Path(pdf_path).name
+ logging.info(f"[{filename}] Processing PDF")
+ try:
+ with pdfplumber.open(pdf_path) as pdf:
+ first_page = pdf.pages[0]
+ last_page = pdf.pages[-1]
+
+ # Try all PDF layers first
+ first_texts = extract_text_from_page_multilayer(first_page, pdf_path, 0)
+ last_texts = extract_text_from_page_multilayer(last_page, pdf_path, len(pdf.pages)-1)
+
+ first_match = find_bates_number(first_texts, pattern)
+ last_match = find_bates_number(last_texts, pattern)
+
+ # If no matches found, try flatten and OCR
+ if not first_match or not last_match:
+ logging.info(f"[{filename}] No matches in text layers, attempting flatten/OCR")
+
+ # For first page
+ if not first_match:
+ try:
+ flattened_text = flatten_and_ocr_page(first_page, pdf_path, 0)
+ if flattened_text:
+ first_texts.append(flattened_text)
+ matches = list(re.finditer(pattern, flattened_text))
+ if matches:
+ first_match = matches[-1]
+ except Exception as e:
+ logging.error(f"[{filename}] Flatten/OCR failed for first page: {e}")
+
+ # For last page
+ if not last_match:
+ try:
+ flattened_text = flatten_and_ocr_page(last_page, pdf_path, len(pdf.pages)-1)
+ if flattened_text:
+ last_texts.append(flattened_text)
+ matches = list(re.finditer(pattern, flattened_text))
+ if matches:
+ last_match = matches[-1]
+ except Exception as e:
+ logging.error(f"[{filename}] Flatten/OCR failed for last page: {e}")
+
+ if first_match and last_match:
+ first_num = ''.join(filter(str.isdigit, first_match.group(0)))
+ last_num = ''.join(filter(str.isdigit, last_match.group(0)))
+
+ logging.info(f"[{filename}] Found numbers: {first_num}β{last_num}")
+ return (first_num, last_num)
+ else:
+ logging.warning(f"[{filename}] No matching numbers found")
+ return None
+ except Exception as e:
+ logging.error(f"[{filename}] Error processing PDF: {str(e)}")
+ return None
+
+def flatten_and_ocr_page(page, pdf_path, page_num):
+ """Flatten page and OCR the crop box area."""
+ filename = Path(pdf_path).name
+ logging.info(f"[{filename}] Page {page_num}: Attempting flatten and OCR")
+
+ try:
+ # Import needed only if we get this far
+ from pdf2image import convert_from_path
+ import pytesseract
+ import PyPDF2
+
+ # Get page dimensions
+ width = page.width
+ height = page.height
+
+ # Calculate crop box for bottom fifth
+ padding = 2
+ y0 = max(0, min(height * 0.8, height - padding))
+ y1 = max(y0 + padding, min(height, height))
+ x0 = padding
+ x1 = max(x0 + padding, min(width - padding, width))
+
+ # Create a single-page PDF with just this page
+ with tempfile.NamedTemporaryFile(suffix='.pdf', delete=False) as tmp_pdf:
+ pdf_writer = PyPDF2.PdfWriter()
+ with open(pdf_path, 'rb') as pdf_file:
+ pdf_reader = PyPDF2.PdfReader(pdf_file)
+ pdf_writer.add_page(pdf_reader.pages[page_num])
+ pdf_writer.write(tmp_pdf)
+ tmp_pdf.flush()
+
+ # Convert to image
+ images = convert_from_path(tmp_pdf.name)
+ if images:
+ # Crop the image to our area of interest
+ img = images[0]
+ img_width, img_height = img.size
+ crop_box_pixels = (
+ int(x0 * img_width / width),
+ int(y0 * img_height / height),
+ int(x1 * img_width / width),
+ int(y1 * img_height / height)
+ )
+ cropped = img.crop(crop_box_pixels)
+
+ # OCR the cropped area
+ text = pytesseract.image_to_string(cropped)
+ if text:
+ logging.info(f"[{filename}] Page {page_num}: Flatten/OCR found: '{text}'")
+ return text
+
+ # Clean up the temporary file
+ os.unlink(tmp_pdf.name)
+
+ except Exception as e:
+ logging.error(f"[{filename}] Page {page_num}: Flatten/OCR failed: {e}")
+ return None
+
+def process_folder(folder_path, pattern, use_ocr, dry_run=False, name_prefix=None):
+ """Process all PDFs in the specified folder."""
+ folder = Path(folder_path)
+ if not folder.exists():
+ logging.error(f"Folder does not exist: {folder_path}")
+ return
+
+ logging.info(f"Processing folder: {folder_path}")
+
+ pdf_count = 0
+ success_count = 0
+ rename_count = 0
+
+ # Use simple case-insensitive matching
+ pdf_files = [f for f in folder.iterdir() if f.is_file() and f.suffix.lower() == '.pdf']
+
+ for pdf_file in pdf_files:
+ pdf_count += 1
+ numbers = extract_bates_numbers(pdf_file, pattern, use_ocr)
+ if numbers:
+ success_count += 1
+ if dry_run:
+ print(f"{pdf_file.name}: {numbers[0]}β{numbers[1]}")
+ elif name_prefix is not None:
+ if rename_with_bates(pdf_file, name_prefix, numbers[0], numbers[1]):
+ rename_count += 1
+
+ logging.info(f"Processed {pdf_count} PDFs, successfully extracted {success_count} number pairs")
+ if not dry_run and name_prefix is not None:
+ logging.info(f"Renamed {rename_count} files")
+
+def main():
+ parser = argparse.ArgumentParser(description='Extract Bates numbers from PDFs')
+ parser.add_argument('folder', help='Path to folder containing PDFs')
+ parser.add_argument('--log', default='INFO',
+ choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'],
+ help='Set the logging level')
+ parser.add_argument('--width-start', type=float, default=0.67,
+ help='Relative x-coordinate to start crop (0-1)')
+ parser.add_argument('--height-start', type=float, default=0.83,
+ help='Relative y-coordinate to start crop (0-1)')
+ parser.add_argument('--prefix', type=str, default='FWS-',
+ help='Prefix pattern to search for (default: "FWS-")')
+ parser.add_argument('--digits', type=int, default=6,
+ help='Number of digits to match after prefix (default: 6)')
+ parser.add_argument('--ocr', action='store_true',
+ help='Enable OCR for pages with little or no text (disabled by default)')
+ parser.add_argument('--dry-run', action='store_true',
+ help='Only print matches without renaming files')
+ parser.add_argument('--name-prefix', type=str,
+ help='Prefix to use when renaming files (e.g., "FWS ")')
+
+ args = parser.parse_args()
+
+ setup_logging(args.log)
+
+ # Check dependencies based on whether OCR is enabled
+ check_dependencies(args.ocr)
+
+ # Display the pattern we're looking for
+ display_pattern = f"{args.prefix}{'#' * args.digits}"
+ print(f"Looking for pattern: {display_pattern}")
+
+ if not args.dry_run and args.name_prefix is None:
+ logging.error("Must specify --name-prefix when not in dry-run mode")
+ sys.exit(1)
+
+ pattern = build_regex_pattern(args.prefix, args.digits)
+ process_folder(args.folder, pattern, args.ocr, args.dry_run, args.name_prefix)
+
+if __name__ == '__main__':
+ main()
+
diff --git a/camel b/camel
new file mode 100755
index 0000000..d14d3db
--- /dev/null
+++ b/camel
@@ -0,0 +1,103 @@
+#!/usr/bin/env python3
+import re
+import os
+import nltk
+from nltk.corpus import words
+from nltk.corpus import wordnet
+
+try:
+ word_list = words.words()
+ nltk.download('wordnet')
+except LookupError:
+ nltk.download('words')
+ nltk.download('wordnet')
+ word_list = words.words()
+
+word_set = set(word.lower() for word in word_list)
+common_words = ['and', 'in', 'the', 'of', 'to', 'at', 'by', 'for', 'with', 'from', 'on']
+always_valid = {'the', 'a', 'an', 'and', 'or', 'but', 'nor', 'for', 'yet', 'so', 'on'}
+
+def is_word(word):
+ if word.lower() in always_valid:
+ print(f" Checking if '{word}' is in dictionary: True (common word)")
+ return True
+
+ in_words = word.lower() in word_set
+ in_wordnet = bool(wordnet.synsets(word))
+ result = in_words and in_wordnet
+ print(f" Checking if '{word}' is in dictionary: {result} (words:{in_words}, wordnet:{in_wordnet})")
+ return result
+
+def process_word(word):
+ print(f"\nProcessing word: '{word}'")
+
+ if is_word(word):
+ print(f" '{word}' is in dictionary, returning as-is")
+ return word
+
+ print(f" '{word}' not in dictionary, checking for common words at end...")
+ for common in common_words:
+ if word.lower().endswith(common):
+ print(f" Found '{common}' at end of '{word}'")
+ remainder = word[:-len(common)]
+ common_case = word[-len(common):]
+ print(f" Recursively processing remainder: '{remainder}'")
+ return f"{process_word(remainder)} {common_case}"
+
+ print(f" No common words found at end of '{word}'")
+
+ match = re.search(r'([a-zA-Z]+)(\d+)$', word)
+ if match:
+ text, num = match.groups()
+ print(f" Found number at end: '{text}' + '{num}'")
+ if is_word(text):
+ return f"{text} {num}"
+
+ print(f" Returning '{word}' unchanged")
+ return word
+
+def split_filename(filename):
+ print(f"\nProcessing filename: {filename}")
+ base = os.path.splitext(filename)[0]
+ ext = os.path.splitext(filename)[1]
+
+ print(f"Splitting on delimiters...")
+ parts = re.split('([_\-\s])', base)
+
+ result = []
+ for part in parts:
+ if part in '_-':
+ result.append(' ')
+ else:
+ print(f"\nSplitting on capitals: {part}")
+ words = re.split('(?<!^)(?=[A-Z])', part)
+ print(f"Got words: {words}")
+ processed = [process_word(word) for word in words]
+ result.append(' '.join(processed))
+
+ final = ' '.join(''.join(result).split())
+ return final + ext
+
+def main():
+ # Get all files in current directory
+ files = [f for f in os.listdir('.') if os.path.isfile(f)]
+
+ for filename in files:
+ if filename.startswith('.'): # Skip hidden files
+ continue
+
+ print(f"\n{'='*50}")
+ print(f"Original: {filename}")
+ new_name = split_filename(filename)
+ print(f"New name: {new_name}")
+
+ if new_name != filename:
+ try:
+ os.rename(filename, new_name)
+ print(f"Renamed: {filename} -> {new_name}")
+ except OSError as e:
+ print(f"Error renaming {filename}: {e}")
+
+if __name__ == "__main__":
+ main()
+
diff --git a/cf b/cf
new file mode 100755
index 0000000..e47b26c
--- /dev/null
+++ b/cf
@@ -0,0 +1,113 @@
+#!/bin/bash
+if [ "$EUID" -ne 0 ]; then
+ echo "This script must be run as root. Try using 'sudo'."
+ exit 1
+fi
+source /home/sij/.zshrc
+source /home/sij/.GLOBAL_VARS
+ddns
+
+# Initialize variables
+full_domain=$1
+shift # Shift the arguments to left so we can get remaining arguments as before
+caddyIP="" # Optional IP for Caddyfile
+port=""
+
+# Fixed IP for Cloudflare from ip.txt
+cloudflareIP=$(cat /home/sij/.services/ip.txt)
+api_key=$CF_API_KEY
+cf_domains_file=/home/sij/.services/cf_domains.json
+
+# Usage message
+usage() {
+ echo "Usage: $0 <full-domain> [--ip <ip address>] --port <port>"
+ echo "Note: <full-domain> is required and can be a subdomain or a full domain."
+ exit 1
+}
+
+# Parse command-line arguments
+while [[ $# -gt 0 ]]; do
+ case $1 in
+ --ip|-i)
+ caddyIP="$2"
+ shift 2
+ ;;
+ --port|-p)
+ port="$2"
+ shift 2
+ ;;
+ *)
+ usage
+ ;;
+ esac
+done
+
+# Check required parameter
+if [[ -z "$full_domain" ]] || [[ -z "$port" ]]; then
+ usage
+fi
+
+# Extract subdomain and domain
+subdomain=$(echo "$full_domain" | awk -F"." '{print $1}')
+remaining_parts=$(echo "$full_domain" | awk -F"." '{print NF}')
+if [ "$remaining_parts" -eq 2 ]; then
+ # Handle root domain (e.g., env.esq)
+ domain=$full_domain
+ subdomain="@" # Use "@" for root domain
+else
+ # Handle subdomain (e.g., sub.env.esq)
+ domain=$(echo "$full_domain" | sed "s/^$subdomain\.//")
+fi
+
+# Default to localhost for Caddyfile if IP is not provided via --ip
+if [[ -z "$caddyIP" ]]; then
+ caddyIP="localhost"
+fi
+
+# Extract zone_id from JSON file
+zone_id=$(jq -r ".\"$domain\".zone_id" "$cf_domains_file")
+
+# Check if zone_id was successfully retrieved
+if [ "$zone_id" == "null" ] || [ -z "$zone_id" ]; then
+ echo "Error: Zone ID for $domain could not be found."
+ exit 1
+fi
+
+# API call setup for Cloudflare A record using the fixed IP from ip.txt
+endpoint="https://api.cloudflare.com/client/v4/zones/$zone_id/dns_records"
+data="{\"type\":\"A\",\"name\":\"$subdomain\",\"content\":\"$cloudflareIP\",\"ttl\":120,\"proxied\":true}"
+
+# Make API call
+response=$(curl -s -X POST "$endpoint" -H "Authorization: Bearer $api_key" -H "Content-Type: application/json" --data "$data")
+
+# Parse response
+record_id=$(echo "$response" | jq -r '.result.id')
+success=$(echo "$response" | jq -r '.success')
+error_message=$(echo "$response" | jq -r '.errors[0].message')
+error_code=$(echo "$response" | jq -r '.errors[0].code')
+
+# Function to update Caddyfile with correct indentation
+update_caddyfile() {
+ echo "$full_domain {
+ reverse_proxy $caddyIP:$port
+ tls {
+ dns cloudflare {env.CLOUDFLARE_API_TOKEN}
+ }
+}" >> /etc/caddy/Caddyfile
+ echo "Configuration appended to /etc/caddy/Caddyfile with correct formatting."
+}
+
+# Check for success or specific error to update Caddyfile
+if [ "$success" == "true" ]; then
+ jq ".\"$domain\".subdomains[\"$full_domain\"] = \"$record_id\"" "$cf_domains_file" > temp.json && mv temp.json "$cf_domains_file"
+ echo "A record created and cf_domains.json updated successfully."
+ update_caddyfile
+elif [ "$error_message" == "Record already exists." ]; then
+ echo "Record already exists. Updating Caddyfile anyway."
+ update_caddyfile
+else
+ echo "Failed to create A record. Error: $error_message (Code: $error_code)"
+fi
+
+echo "Restarting caddy!"
+sudo systemctl restart caddy
diff --git a/checknode b/checknode
new file mode 100755
index 0000000..327a6e6
--- /dev/null
+++ b/checknode
@@ -0,0 +1,55 @@
+#!/bin/bash
+
+echo "Checking for remnants of Node.js, npm, and nvm..."
+
+# Check PATH
+echo "Checking PATH..."
+echo $PATH | grep -q 'node\|npm' && echo "Found Node.js or npm in PATH"
+
+# Check Homebrew
+echo "Checking Homebrew..."
+brew list | grep -q 'node\|npm' && echo "Found Node.js or npm installed with Homebrew"
+
+# Check Yarn
+echo "Checking Yarn..."
+command -v yarn >/dev/null 2>&1 && echo "Found Yarn"
+
+# Check Node.js and npm directories
+echo "Checking Node.js and npm directories..."
+ls ~/.npm >/dev/null 2>&1 && echo "Found ~/.npm directory"
+ls ~/.node-gyp >/dev/null 2>&1 && echo "Found ~/.node-gyp directory"
+
+# Check open files and sockets
+echo "Checking open files and sockets..."
+lsof | grep -q 'node' && echo "Found open files or sockets related to Node.js"
+
+# Check other version managers
+echo "Checking other version managers..."
+command -v n >/dev/null 2>&1 && echo "Found 'n' version manager"
+
+# Check temporary directories
+echo "Checking temporary directories..."
+ls /tmp | grep -q 'node\|npm' && echo "Found Node.js or npm related files in /tmp"
+
+# Check Browserify and Webpack cache
+echo "Checking Browserify and Webpack cache..."
+ls ~/.config/browserify >/dev/null 2>&1 && echo "Found Browserify cache"
+ls ~/.config/webpack >/dev/null 2>&1 && echo "Found Webpack cache"
+
+# Check Electron cache
+echo "Checking Electron cache..."
+ls ~/.electron >/dev/null 2>&1 && echo "Found Electron cache"
+
+# Check logs
+echo "Checking logs..."
+ls ~/.npm/_logs >/dev/null 2>&1 && echo "Found npm logs"
+ls ~/.node-gyp/*.log >/dev/null 2>&1 && echo "Found Node.js logs"
+
+# Check miscellaneous directories
+echo "Checking miscellaneous directories..."
+ls ~/.node_repl_history >/dev/null 2>&1 && echo "Found ~/.node_repl_history"
+ls ~/.v8flags* >/dev/null 2>&1 && echo "Found ~/.v8flags*"
+ls ~/.npm-global >/dev/null 2>&1 && echo "Found ~/.npm-global"
+ls ~/.nvm-global >/dev/null 2>&1 && echo "Found ~/.nvm-global"
+
+echo "Check completed."
diff --git a/comfy b/comfy
new file mode 100755
index 0000000..3093b61
--- /dev/null
+++ b/comfy
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+# Create a new tmux session named "comfy" detached (-d) and start the first command in the left pane
+tmux new-session -d -s comfy -n comfypane
+
+# Split the window into two panes. By default, this creates a vertical split.
+tmux split-window -h -t comfy
+
+# Select the first pane to setup comfy environment
+tmux select-pane -t 0
+COMFY_MAMBA=$(mamba env list | grep "^comfy" | awk '{print $2}')
+tmux send-keys -t 0 "cd ~/workshop/sd/ComfyUI" C-m
+tmux send-keys -t 0 "export PATH=\"$COMFY_MAMBA/bin:\$PATH\"" C-m
+tmux send-keys -t 0 "source ~/.zshrc" C-m
+tmux send-keys -t 0 "mamba activate comfy; sleep 1; while true; do PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 PYTORCH_ENABLE_MPS_FALLBACK=1 python main.py --preview-method auto --force-fp16 --enable-cors-header; exit_status=\$?; if [ \$exit_status -ne 0 ]; then osascript -e 'display notification \"ComfyUI script exited unexpectedly\" with title \"Error in ComfyUI\"'; fi; sleep 1; done" C-m
+
+# Select the second pane to setup extracomfy environment
+tmux select-pane -t 1
+IG_MAMBA=$(mamba env list | grep "^insta" | awk '{print $2}')
+tmux send-keys -t 1 "export PATH=\"$IG_MAMBA/bin:\$PATH\"" C-m
+tmux send-keys -t 1 "source ~/.zshrc" C-m
+tmux send-keys -t 1 "mamba activate instabot; cd workshop/igbot" C-m
+
+# Attach to the tmux session
+# tmux attach -t comfy
+
diff --git a/ddns b/ddns
new file mode 100755
index 0000000..6a78dff
--- /dev/null
+++ b/ddns
@@ -0,0 +1,55 @@
+#!/bin/bash
+source /home/sij/.GLOBAL_VARS
+
+service="https://am.i.mullvad.net/ip"
+# Obtain the current public IP address
+#current_ip=$(ssh -n sij@10.13.37.10 curl -s $service)
+current_ip=$(curl -s $service)
+last_ip=$(cat /home/sij/.services/ip.txt)
+api_token=$CF_API_KEY
+
+# Path to the JSON file with zone IDs, subdomains, and DNS IDs mappings
+json_file="/home/sij/.services/cf_domains.json"
+
+force_update=false
+
+# Parse command line arguments for --force flag
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -f|--force) force_update=true ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+# Temporary file to store update results
+temp_file=$(mktemp)
+
+# Function to update DNS records
+update_dns_record() {
+ zone_id=$1
+ subdomain=$2
+ dns_id=$3
+ update_result=$(curl -s -X PUT "https://api.cloudflare.com/client/v4/zones/$zone_id/dns_records/$dns_id" \
+ -H "Authorization: Bearer $api_token" \
+ -H "Content-Type: application/json" \
+ --data "{\"type\":\"A\",\"name\":\"$subdomain\",\"content\":\"$current_ip\",\"ttl\":120,\"proxied\":true}")
+ echo "$update_result" >> "$temp_file"
+}
+
+# Check if IP has changed or --force flag is used
+if [ "$current_ip" != "$last_ip" ] || [ "$force_update" = true ]; then
+ echo $current_ip > /home/sij/.services/ip.txt
+ # Iterate through each domain in the JSON
+ /home/sij/miniforge3/bin/jq -r '.[] | .zone_id as $zone_id | .subdomains | to_entries[] | [$zone_id, .key, .value] | @tsv' $json_file |
+ while IFS=$'\t' read -r zone_id subdomain dns_id; do
+ update_dns_record "$zone_id" "$subdomain" "$dns_id"
+ done
+ # Combine all update results into a single JSON array
+ /home/sij/miniforge3/bin/jq -s '.' "$temp_file"
+ # Remove the temporary file
+ rm "$temp_file"
+else
+ echo "IP address has not changed from ${last_ip}. No action taken."
+fi
+
diff --git a/delpycache b/delpycache
new file mode 100755
index 0000000..5ec9a3a
--- /dev/null
+++ b/delpycache
@@ -0,0 +1,27 @@
+#!/bin/bash
+
+# Default directories to search
+directories=("~/sync" "~/workshop")
+
+# Check if a command line argument is provided
+if [ $# -gt 0 ]; then
+ if [ "$1" == "." ]; then
+ # Use the current directory
+ directories=(".")
+ else
+ # Use the provided directory
+ directories=("$1")
+ fi
+fi
+
+# Iterate through each directory
+for dir in "${directories[@]}"; do
+ # Expand tilde to home directory
+ expanded_dir=$(eval echo "$dir")
+
+ # Find and delete __pycache__ directories
+ find "$expanded_dir" -type d -name "__pycache__" -exec rm -rf {} +
+done
+
+echo "Deletion of __pycache__ folders completed."
+
diff --git a/deps b/deps
new file mode 100755
index 0000000..fa32adc
--- /dev/null
+++ b/deps
@@ -0,0 +1,429 @@
+#!/usr/bin/env python3
+
+import argparse
+import os
+import re
+import subprocess
+import sys
+import urllib.request
+import urllib.error
+
+############################
+# Built-in, Known Corrections, Exclusions
+############################
+
+BUILTIN_MODULES = {
+ 'abc', 'aifc', 'argparse', 'array', 'ast', 'asynchat', 'asyncio', 'asyncore', 'atexit',
+ 'audioop', 'base64', 'bdb', 'binascii', 'binhex', 'bisect', 'builtins', 'bz2', 'calendar',
+ 'cgi', 'cgitb', 'chunk', 'cmath', 'cmd', 'code', 'codecs', 'codeop', 'collections', 'colorsys',
+ 'compileall', 'concurrent', 'configparser', 'contextlib', 'copy', 'copyreg', 'crypt', 'csv',
+ 'ctypes', 'curses', 'dataclasses', 'datetime', 'dbm', 'decimal', 'difflib', 'dis', 'distutils',
+ 'doctest', 'dummy_threading', 'email', 'encodings', 'ensurepip', 'enum', 'errno', 'faulthandler',
+ 'fcntl', 'filecmp', 'fileinput', 'fnmatch', 'formatter', 'fractions', 'ftplib', 'functools',
+ 'gc', 'getopt', 'getpass', 'gettext', 'glob', 'gzip', 'hashlib', 'heapq', 'hmac', 'html', 'http',
+ 'imaplib', 'imghdr', 'imp', 'importlib', 'inspect', 'io', 'ipaddress', 'itertools', 'json',
+ 'keyword', 'lib2to3', 'linecache', 'locale', 'logging', 'lzma', 'mailbox', 'mailcap', 'marshal',
+ 'math', 'mimetypes', 'modulefinder', 'multiprocessing', 'netrc', 'nntplib', 'numbers', 'operator',
+ 'optparse', 'os', 'ossaudiodev', 'parser', 'pathlib', 'pdb', 'pickle', 'pickletools', 'pipes',
+ 'pkgutil', 'platform', 'plistlib', 'poplib', 'posix', 'pprint', 'profile', 'pstats', 'pty',
+ 'pwd', 'py_compile', 'pyclbr', 'pydoc', 'queue', 'quopri', 'random', 're', 'readline',
+ 'reprlib', 'resource', 'rlcompleter', 'runpy', 'sched', 'secrets', 'select', 'selectors', 'shelve',
+ 'shlex', 'shutil', 'signal', 'site', 'smtpd', 'smtplib', 'sndhdr', 'socket', 'socketserver',
+ 'spwd', 'sqlite3', 'ssl', 'stat', 'statistics', 'string', 'stringprep', 'struct', 'subprocess',
+ 'sunau', 'symtable', 'sys', 'sysconfig', 'syslog', 'tabnanny', 'tarfile', 'telnetlib', 'tempfile',
+ 'termios', 'test', 'textwrap', 'threading', 'time', 'timeit', 'token', 'tokenize', 'trace',
+ 'traceback', 'tracemalloc', 'tty', 'turtle', 'types', 'typing', 'unicodedata', 'unittest',
+ 'urllib', 'uu', 'uuid', 'venv', 'warnings', 'wave', 'weakref', 'webbrowser', 'xdrlib', 'xml',
+ 'xmlrpc', 'zipapp', 'zipfile', 'zipimport', 'zlib'
+}
+
+KNOWN_CORRECTIONS = {
+ 'dateutil': 'python-dateutil',
+ 'dotenv': 'python-dotenv',
+ 'docx': 'python-docx',
+ 'tesseract': 'pytesseract',
+ 'magic': 'python-magic',
+ 'multipart': 'python-multipart',
+ 'newspaper': 'newspaper3k',
+ 'srtm': 'elevation',
+ 'yaml': 'pyyaml',
+ 'zoneinfo': 'backports.zoneinfo'
+}
+
+EXCLUDED_NAMES = {'models', 'data', 'convert', 'example', 'tests'}
+
+############################
+# Environment & Installation
+############################
+
+def run_command(command):
+ process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ stdout, stderr = process.communicate()
+ return process.returncode, stdout.decode(), stderr.decode()
+
+def which(cmd):
+ """
+ Check if `cmd` is on PATH. Returns True if found, else False.
+ """
+ for pth in os.environ["PATH"].split(os.pathsep):
+ cmd_path = os.path.join(pth, cmd)
+ if os.path.isfile(cmd_path) and os.access(cmd_path, os.X_OK):
+ return True
+ return False
+
+def in_conda_env():
+ """
+ Returns True if we appear to be in a conda environment,
+ typically indicated by CONDA_DEFAULT_ENV or other variables.
+ """
+ return "CONDA_DEFAULT_ENV" in os.environ
+
+# We'll detect once at runtime (if in a conda env and skip_conda=False):
+# we either pick 'mamba' if available, else 'conda' if available, else None
+PREFERRED_CONDA_TOOL = None
+
+def detect_conda_tool(skip_conda=False):
+ """
+ Decide which tool to use for conda-based installation:
+ 1) If skip_conda is True or not in a conda env -> return None
+ 2) If mamba is installed, return 'mamba'
+ 3) Else if conda is installed, return 'conda'
+ 4) Else return None
+ """
+ if skip_conda or not in_conda_env():
+ return None
+ if which("mamba"):
+ return "mamba"
+ elif which("conda"):
+ return "conda"
+ return None
+
+def is_package_installed(package, skip_conda=False):
+ """
+ Checks if 'package' is installed with the chosen conda tool or pip.
+ """
+ conda_tool = detect_conda_tool(skip_conda)
+ if conda_tool == "mamba":
+ returncode, stdout, _ = run_command(["mamba", "list"])
+ if returncode == 0:
+ pattern = rf"^{re.escape(package)}\s"
+ if re.search(pattern, stdout, re.MULTILINE):
+ return True
+ elif conda_tool == "conda":
+ returncode, stdout, _ = run_command(["conda", "list"])
+ if returncode == 0:
+ pattern = rf"^{re.escape(package)}\s"
+ if re.search(pattern, stdout, re.MULTILINE):
+ return True
+
+ # Fall back to pip
+ returncode, stdout, _ = run_command(["pip", "list"])
+ pattern = rf"^{re.escape(package)}\s"
+ return re.search(pattern, stdout, re.MULTILINE) is not None
+
+def install_package(package, skip_conda=False):
+ """
+ Installs 'package'.
+ 1) Decide once if we can use 'mamba' or 'conda' (if skip_conda=False and in conda env).
+ 2) Try that conda tool for installation
+ 3) If that fails or not found, fallback to pip
+ """
+ if is_package_installed(package, skip_conda=skip_conda):
+ print(f"Package '{package}' is already installed.")
+ return
+
+ conda_tool = detect_conda_tool(skip_conda)
+
+ if conda_tool == "mamba":
+ print(f"Installing '{package}' with mamba...")
+ returncode, _, _ = run_command(["mamba", "install", "-y", "-c", "conda-forge", package])
+ if returncode == 0:
+ print(f"Successfully installed '{package}' via mamba.")
+ return
+ print(f"mamba failed for '{package}'. Falling back to pip...")
+
+ elif conda_tool == "conda":
+ print(f"Installing '{package}' with conda...")
+ returncode, _, _ = run_command(["conda", "install", "-y", "-c", "conda-forge", package])
+ if returncode == 0:
+ print(f"Successfully installed '{package}' via conda.")
+ return
+ print(f"conda failed for '{package}'. Falling back to pip...")
+
+ # fallback: pip
+ print(f"Installing '{package}' with pip...")
+ returncode, _, _ = run_command(["pip", "install", package])
+ if returncode != 0:
+ print(f"Failed to install package '{package}'.")
+ else:
+ print(f"Successfully installed '{package}' via pip.")
+
+############################
+# Parsing Python Imports
+############################
+
+def process_requirements_file(file_path):
+ packages = set()
+ with open(file_path, 'r') as file:
+ for line in file:
+ line = line.strip()
+ if line and not line.startswith('#'):
+ packages.add(line)
+ return packages
+
+def process_python_file(file_path):
+ """
+ Return a set of external imports (not built-in or excluded).
+ Applies known corrections to recognized package names.
+ """
+ imports = set()
+ with open(file_path, 'r', encoding='utf-8', errors='ignore') as f:
+ content = f.read()
+
+ for line in content.split('\n'):
+ line = line.strip()
+ if line.startswith(('import ', 'from ')) and not line.startswith('#'):
+ if line.startswith('import '):
+ modules = line.replace('import ', '').split(',')
+ for mod in modules:
+ mod = re.sub(r'\s+as\s+\w+', '', mod).split('.')[0].strip()
+ if mod and not mod.isupper() and mod not in EXCLUDED_NAMES and mod not in BUILTIN_MODULES:
+ imports.add(KNOWN_CORRECTIONS.get(mod, mod))
+ elif line.startswith('from '):
+ mod = line.split(' ')[1].split('.')[0].strip()
+ if mod and not mod.isupper() and mod not in EXCLUDED_NAMES and mod not in BUILTIN_MODULES:
+ imports.add(KNOWN_CORRECTIONS.get(mod, mod))
+
+ return imports
+
+def find_imports_in_path(path, recurse=False):
+ """
+ Finds Python imports in the specified path. If path is a file, parse that file;
+ if path is a dir, parse .py files in that dir. Recurse subdirs if 'recurse=True'.
+ """
+ imports = set()
+ if not os.path.exists(path):
+ print(f"Warning: Path does not exist: {path}")
+ return imports
+
+ if os.path.isfile(path):
+ if path.endswith('.py'):
+ imports.update(process_python_file(path))
+ else:
+ print(f"Skipping non-Python file: {path}")
+ return imports
+
+ # Directory:
+ if recurse:
+ for root, _, filenames in os.walk(path):
+ for fn in filenames:
+ if fn.endswith('.py'):
+ imports.update(process_python_file(os.path.join(root, fn)))
+ else:
+ for fn in os.listdir(path):
+ fullpath = os.path.join(path, fn)
+ if os.path.isfile(fullpath) and fn.endswith('.py'):
+ imports.update(process_python_file(fullpath))
+
+ return imports
+
+############################
+# PyPI Availability Check
+############################
+
+def check_library_on_pypi(library):
+ """
+ Returns True if 'library' is on PyPI, else False.
+ Using urllib to avoid external dependencies.
+ """
+ url = f"https://pypi.org/pypi/{library}/json"
+ try:
+ with urllib.request.urlopen(url, timeout=5) as resp:
+ return (resp.status == 200) # 200 => available
+ except (urllib.error.URLError, urllib.error.HTTPError, ValueError):
+ return False
+
+############################
+# Writing to requirements/missing
+############################
+
+def append_to_file(line, filename):
+ """
+ Append 'line' to 'filename' only if it's not already in there.
+ """
+ if not os.path.isfile(filename):
+ with open(filename, 'w') as f:
+ f.write(line + '\n')
+ return
+
+ with open(filename, 'r') as f:
+ lines = {l.strip() for l in f.readlines() if l.strip()}
+ if line not in lines:
+ with open(filename, 'a') as f:
+ f.write(line + '\n')
+
+############################
+# Subcommand: ls
+############################
+
+def subcmd_ls(parsed_args):
+ """
+ Gathers imports, displays them, then writes them to requirements.txt or missing-packages.txt
+ just like the original import_finder script did.
+ """
+ path = parsed_args.path
+ recurse = parsed_args.recurse
+
+ imports = find_imports_in_path(path, recurse=recurse)
+ if not imports:
+ print("No Python imports found (or none that require external packages).")
+ return
+
+ print("Imports found:")
+ for imp in sorted(imports):
+ print(f" - {imp}")
+
+ # Now we replicate the logic of import_finder:
+ # If on PyPI => requirements.txt; else => missing-packages.txt
+ for lib in sorted(imports):
+ if check_library_on_pypi(lib):
+ append_to_file(lib, 'requirements.txt')
+ else:
+ append_to_file(lib, 'missing-packages.txt')
+
+ print("\nWrote results to requirements.txt (PyPI-available) and missing-packages.txt (unavailable).")
+
+############################
+# Subcommand: install
+############################
+
+def subcmd_install(parsed_args):
+ """
+ If the user typed no direct packages/scripts or only used '-r' for recursion with no other args,
+ we gather imports from the current dir, check PyPI availability, and install them with conda/mamba/pip
+ (unless --no-conda is given).
+
+ Otherwise, if the user typed e.g. '-R <reqfile>', or a .py file, or direct package names, we handle them.
+ """
+ skip_conda = parsed_args.no_conda
+ is_recursive = parsed_args.recurse
+
+ # If user typed no leftover arguments or only the recursion flag, we do the "auto-scan & install" mode
+ if not parsed_args.packages:
+ # Means: "deps install" or "deps install -r"
+ imports_found = find_imports_in_path('.', recurse=is_recursive)
+ if not imports_found:
+ print("No imports found in current directory.")
+ return
+ # Filter out those that are on PyPI
+ to_install = []
+ for lib in sorted(imports_found):
+ if check_library_on_pypi(lib):
+ to_install.append(lib)
+ else:
+ print(f"Skipping '{lib}' (not found on PyPI).")
+ if not to_install:
+ print("No PyPI-available packages found to install.")
+ return
+ print("Installing packages:", ', '.join(to_install))
+ for pkg in to_install:
+ install_package(pkg, skip_conda=skip_conda)
+ return
+
+ # Otherwise, we have leftover items: direct packages, .py files, or "-R" requirements.
+ leftover_args = parsed_args.packages
+ packages_to_install = set()
+
+ i = 0
+ while i < len(leftover_args):
+ arg = leftover_args[i]
+ if arg == '-R':
+ # next arg is a requirements file
+ if i + 1 < len(leftover_args):
+ req_file = leftover_args[i + 1]
+ if os.path.isfile(req_file):
+ pkgs = process_requirements_file(req_file)
+ packages_to_install.update(pkgs)
+ else:
+ print(f"Requirements file not found: {req_file}")
+ i += 2
+ else:
+ print("Error: -R requires a file path.")
+ return
+ elif arg.endswith('.py'):
+ # parse imports from that script
+ if os.path.isfile(arg):
+ pkgs = process_python_file(arg)
+ packages_to_install.update(pkgs)
+ else:
+ print(f"File not found: {arg}")
+ i += 1
+ else:
+ # treat as a direct package name
+ packages_to_install.add(arg)
+ i += 1
+
+ # Now install them
+ for pkg in sorted(packages_to_install):
+ install_package(pkg, skip_conda=skip_conda)
+
+############################
+# Main
+############################
+
+def main():
+ parser = argparse.ArgumentParser(description='deps - Manage and inspect Python dependencies.')
+ subparsers = parser.add_subparsers(dest='subcommand', required=True)
+
+ # Subcommand: ls
+ ls_parser = subparsers.add_parser(
+ 'ls',
+ help="List imports in a file/folder (and write them to requirements.txt/missing-packages.txt)."
+ )
+ ls_parser.add_argument(
+ '-r', '--recurse',
+ action='store_true',
+ help='Recurse into subfolders.'
+ )
+ ls_parser.add_argument(
+ 'path',
+ nargs='?',
+ default='.',
+ help='File or directory to scan (default is current directory).'
+ )
+ ls_parser.set_defaults(func=subcmd_ls)
+
+ # Subcommand: install
+ install_parser = subparsers.add_parser(
+ 'install',
+ help="Install packages or dependencies from .py files / current folder / subfolders."
+ )
+ install_parser.add_argument(
+ '-r', '--recurse',
+ action='store_true',
+ help="If no packages are specified, scanning current dir for imports will be recursive."
+ )
+ install_parser.add_argument(
+ '--no-conda',
+ action='store_true',
+ help="Skip using mamba/conda entirely and install only with pip."
+ )
+ install_parser.add_argument(
+ 'packages',
+ nargs='*',
+ help=(
+ "Direct package names, .py files, or '-R <reqfile>'. If empty, scans current dir; "
+ "if combined with -r, scans recursively. Example usage:\n"
+ " deps install requests flask\n"
+ " deps install script.py\n"
+ " deps install -R requirements.txt\n"
+ " deps install -r (recursively scan current dir)\n"
+ )
+ )
+ install_parser.set_defaults(func=subcmd_install)
+
+ parsed_args = parser.parse_args()
+ parsed_args.func(parsed_args)
+
+if __name__ == "__main__":
+ main()
diff --git a/emoji_flag b/emoji_flag
new file mode 100755
index 0000000..eba028f
--- /dev/null
+++ b/emoji_flag
@@ -0,0 +1,14 @@
+#!/usr/bin/env python3
+import sys
+
+def flag_emoji(country_code):
+ offset = 127397
+ flag = ''.join(chr(ord(char) + offset) for char in country_code.upper())
+ return flag
+
+if __name__ == "__main__":
+ if len(sys.argv) > 1:
+ country_code = sys.argv[1]
+ print(flag_emoji(country_code))
+ else:
+ print("No country code provided")
\ No newline at end of file
diff --git a/get b/get
new file mode 100755
index 0000000..f7ab37e
--- /dev/null
+++ b/get
@@ -0,0 +1,51 @@
+#!/bin/bash
+
+# Check if a URL is provided
+if [ $# -eq 0 ]; then
+ echo "Please provide a git repository URL."
+ exit 1
+fi
+
+# Extract the repository URL and name
+repo_url=$1
+repo_name=$(basename "$repo_url" .git)
+
+# Clone the repository
+git clone "$repo_url"
+
+# Check if the clone was successful
+if [ $? -ne 0 ]; then
+ echo "Failed to clone the repository."
+ exit 1
+fi
+
+# Change to the newly created directory
+cd "$repo_name" || exit
+
+# Check for setup.py or requirements.txt
+if [ -f "setup.py" ] || [ -f "requirements.txt" ]; then
+ # Create a new Mamba environment
+ mamba create -n "$repo_name" python -y
+
+ # Activate the new environment
+ eval "$(conda shell.bash hook)"
+ mamba activate "$repo_name"
+
+ # Install dependencies
+ if [ -f "setup.py" ]; then
+ echo "Installing from setup.py..."
+ python setup.py install
+ fi
+
+ if [ -f "requirements.txt" ]; then
+ echo "Installing from requirements.txt..."
+ pip install -r requirements.txt
+ fi
+
+ echo "Environment setup complete."
+else
+ echo "No setup.py or requirements.txt found. Skipping environment setup."
+fi
+
+echo "Repository cloned and set up successfully."
+
diff --git a/gitpurge b/gitpurge
new file mode 100755
index 0000000..5831a9d
--- /dev/null
+++ b/gitpurge
@@ -0,0 +1,33 @@
+#!/bin/bash
+
+# Ensure we're in a git repository
+if ! git rev-parse --is-inside-work-tree > /dev/null 2>&1; then
+ echo "Error: This script must be run inside a Git repository."
+ exit 1
+fi
+
+# Check if git-filter-repo is installed
+if ! command -v git-filter-repo &> /dev/null; then
+ echo "Error: git-filter-repo is not installed. Please install it first."
+ echo "You can install it via pip: pip install git-filter-repo"
+ exit 1
+fi
+
+# Get a list of files that currently exist in the repository
+current_files=$(git ls-files)
+
+# Create a file with the list of files to keep
+echo "$current_files" > files_to_keep.txt
+
+# Use git-filter-repo to keep only the files that currently exist
+git filter-repo --paths-from-file files_to_keep.txt --force
+
+# Remove the temporary file
+rm files_to_keep.txt
+
+# Force push all branches
+git push origin --all --force
+
+echo "Purge complete. All files not present in the local repo have been removed from all commits on all branches."
+echo "The changes have been force-pushed to the remote repository."
+
diff --git a/gitscan b/gitscan
new file mode 100755
index 0000000..c887990
--- /dev/null
+++ b/gitscan
@@ -0,0 +1,22 @@
+#!/bin/bash
+
+output_file="./repos.txt"
+
+# Clear the existing file or create it if it doesn't exist
+> "$output_file"
+
+# Find all .git directories in the current folder and subfolders,
+# excluding hidden directories and suppressing permission denied errors
+find . -type d -name ".git" -not -path "*/.*/*" 2>/dev/null | while read -r gitdir; do
+ # Get the parent directory of the .git folder
+ repo_path=$(dirname "$gitdir")
+ echo "$repo_path" >> "$output_file"
+done
+
+echo "Git repositories have been written to $output_file"
+
+# Remove duplicate entries
+sort -u "$output_file" -o "$output_file"
+
+echo "Duplicate entries removed. Final list:"
+cat "$output_file"
\ No newline at end of file
diff --git a/import_finder b/import_finder
new file mode 100755
index 0000000..41cbf6b
--- /dev/null
+++ b/import_finder
@@ -0,0 +1,144 @@
+#!/usr/bin/env python3
+
+import os
+import re
+import requests
+import time
+import pkg_resources
+
+# List of Python built-in modules
+BUILTIN_MODULES = {
+ 'abc', 'aifc', 'argparse', 'array', 'ast', 'asynchat', 'asyncio', 'asyncore', 'atexit',
+ 'audioop', 'base64', 'bdb', 'binascii', 'binhex', 'bisect', 'builtins', 'bz2', 'calendar',
+ 'cgi', 'cgitb', 'chunk', 'cmath', 'cmd', 'code', 'codecs', 'codeop', 'collections', 'colorsys',
+ 'compileall', 'concurrent', 'configparser', 'contextlib', 'copy', 'copyreg', 'crypt', 'csv',
+ 'ctypes', 'curses', 'dataclasses', 'datetime', 'dbm', 'decimal', 'difflib', 'dis', 'distutils',
+ 'doctest', 'dummy_threading', 'email', 'encodings', 'ensurepip', 'enum', 'errno', 'faulthandler',
+ 'fcntl', 'filecmp', 'fileinput', 'fnmatch', 'formatter', 'fractions', 'ftplib', 'functools',
+ 'gc', 'getopt', 'getpass', 'gettext', 'glob', 'gzip', 'hashlib', 'heapq', 'hmac', 'html', 'http',
+ 'imaplib', 'imghdr', 'imp', 'importlib', 'inspect', 'io', 'ipaddress', 'itertools', 'json',
+ 'keyword', 'lib2to3', 'linecache', 'locale', 'logging', 'lzma', 'mailbox', 'mailcap', 'marshal',
+ 'math', 'mimetypes', 'modulefinder', 'multiprocessing', 'netrc', 'nntplib', 'numbers', 'operator',
+ 'optparse', 'os', 'ossaudiodev', 'parser', 'pathlib', 'pdb', 'pickle', 'pickletools', 'pipes',
+ 'pkgutil', 'platform', 'plistlib', 'poplib', 'posix', 'pprint', 'profile', 'pstats', 'pty',
+ 'pwd', 'py_compile', 'pyclbr', 'pydoc', 'queue', 'quopri', 'random', 're', 'readline',
+ 'reprlib', 'resource', 'rlcompleter', 'runpy', 'sched', 'secrets', 'select', 'selectors', 'shelve',
+ 'shlex', 'shutil', 'signal', 'site', 'smtpd', 'smtplib', 'sndhdr', 'socket', 'socketserver',
+ 'spwd', 'sqlite3', 'ssl', 'stat', 'statistics', 'string', 'stringprep', 'struct', 'subprocess',
+ 'sunau', 'symtable', 'sys', 'sysconfig', 'syslog', 'tabnanny', 'tarfile', 'telnetlib', 'tempfile',
+ 'termios', 'test', 'textwrap', 'threading', 'time', 'timeit', 'token', 'tokenize', 'trace',
+ 'traceback', 'tracemalloc', 'tty', 'turtle', 'types', 'typing', 'unicodedata', 'unittest',
+ 'urllib', 'uu', 'uuid', 'venv', 'warnings', 'wave', 'weakref', 'webbrowser', 'xdrlib', 'xml',
+ 'xmlrpc', 'zipapp', 'zipfile', 'zipimport', 'zlib'
+}
+
+# Known corrections for PyPI package names
+KNOWN_CORRECTIONS = {
+ 'dateutil': 'python-dateutil',
+ 'dotenv': 'python-dotenv',
+ 'docx': 'python-docx',
+ 'tesseract': 'pytesseract',
+ 'magic': 'python-magic',
+ 'multipart': 'python-multipart',
+ 'newspaper': 'newspaper3k',
+ 'srtm': 'elevation',
+ 'yaml': 'pyyaml',
+ 'zoneinfo': 'backports.zoneinfo'
+}
+
+# List of generic names to exclude
+EXCLUDED_NAMES = {'models', 'data', 'convert', 'example', 'tests'}
+
+def find_imports(root_dir):
+ imports_by_file = {}
+ for dirpath, _, filenames in os.walk(root_dir):
+ for filename in filenames:
+ if filename.endswith('.py'):
+ filepath = os.path.join(dirpath, filename)
+ with open(filepath, 'r') as file:
+ import_lines = []
+ for line in file:
+ line = line.strip()
+ if line.startswith(('import ', 'from ')) and not line.startswith('#'):
+ import_lines.append(line)
+ imports_by_file[filepath] = import_lines
+ return imports_by_file
+
+def process_import_lines(import_lines):
+ processed_lines = set() # Use a set to remove duplicates
+ for line in import_lines:
+ # Handle 'import xyz' and 'import abc, def, geh'
+ if line.startswith('import '):
+ modules = line.replace('import ', '').split(',')
+ for mod in modules:
+ mod = re.sub(r'\s+as\s+\w+', '', mod).split('.')[0].strip()
+ if mod and not mod.isupper() and mod not in EXCLUDED_NAMES:
+ processed_lines.add(mod)
+ # Handle 'from abc import def, geh'
+ elif line.startswith('from '):
+ mod = line.split(' ')[1].split('.')[0].strip()
+ if mod and not mod.isupper() and mod not in EXCLUDED_NAMES:
+ processed_lines.add(mod)
+ return processed_lines
+
+def check_pypi_availability(libraries):
+ available = set()
+ unavailable = set()
+ for lib in libraries:
+ if lib in BUILTIN_MODULES: # Skip built-in modules
+ continue
+ corrected_lib = KNOWN_CORRECTIONS.get(lib, lib)
+ try:
+ if check_library_on_pypi(corrected_lib):
+ available.add(corrected_lib)
+ else:
+ unavailable.add(corrected_lib)
+ except requests.exceptions.RequestException:
+ print(f"Warning: Unable to check {corrected_lib} on PyPI due to network error.")
+ unavailable.add(corrected_lib)
+ return available, unavailable
+
+def check_library_on_pypi(library):
+ max_retries = 3
+ for attempt in range(max_retries):
+ try:
+ response = requests.get(f"https://pypi.org/pypi/{library}/json", timeout=5)
+ return response.status_code == 200
+ except requests.exceptions.RequestException:
+ if attempt < max_retries - 1:
+ time.sleep(1) # Wait for 1 second before retrying
+ else:
+ raise
+
+def save_to_requirements_file(available, output_file='requirements.txt'):
+ existing_requirements = set()
+ if os.path.isfile(output_file):
+ with open(output_file, 'r') as file:
+ existing_requirements = set(line.strip() for line in file)
+ with open(output_file, 'a') as file:
+ for pkg in sorted(available - existing_requirements):
+ print(f"Adding to requirements.txt: {pkg}")
+ file.write(pkg + '\n')
+
+def save_to_missing_file(unavailable, output_file='missing-packages.txt'):
+ existing_missing = set()
+ if os.path.isfile(output_file):
+ with open(output_file, 'r') as file:
+ existing_missing = set(line.strip() for line in file)
+ with open(output_file, 'a') as file:
+ for pkg in sorted(unavailable - existing_missing):
+ print(f"Adding to missing-packages.txt: {pkg}")
+ file.write(pkg + '\n')
+
+if __name__ == "__main__":
+ root_dir = os.getcwd() # Get the current working directory
+
+ imports_by_file = find_imports(root_dir)
+ for filepath, import_lines in imports_by_file.items():
+ print(f"# Processing {filepath}")
+ processed_lines = process_import_lines(import_lines)
+ available, unavailable = check_pypi_availability(processed_lines)
+ save_to_requirements_file(available)
+ save_to_missing_file(unavailable)
+
+ print(f"Processed import statements have been saved to requirements.txt and missing-packages.txt")
diff --git a/ip b/ip
new file mode 100755
index 0000000..831d81d
--- /dev/null
+++ b/ip
@@ -0,0 +1,2 @@
+#!/bin/bash
+whois $(curl -s https://am.i.mullvad.net/ip)
diff --git a/kip b/kip
new file mode 100755
index 0000000..5946d92
--- /dev/null
+++ b/kip
@@ -0,0 +1,144 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+import re
+import subprocess
+import requests
+import time
+from typing import Set, Tuple, List, Dict
+
+# List of Python built-in modules
+BUILTIN_MODULES = {
+ 'abc', 'aifc', 'argparse', 'array', 'ast', 'asynchat', 'asyncio', 'asyncore', 'atexit',
+ 'audioop', 'base64', 'bdb', 'binascii', 'binhex', 'bisect', 'builtins', 'bz2', 'calendar',
+ 'cgi', 'cgitb', 'chunk', 'cmath', 'cmd', 'code', 'codecs', 'codeop', 'collections', 'colorsys',
+ 'compileall', 'concurrent', 'configparser', 'contextlib', 'copy', 'copyreg', 'crypt', 'csv',
+ 'ctypes', 'curses', 'dataclasses', 'datetime', 'dbm', 'decimal', 'difflib', 'dis', 'distutils',
+ 'doctest', 'dummy_threading', 'email', 'encodings', 'ensurepip', 'enum', 'errno', 'faulthandler',
+ 'fcntl', 'filecmp', 'fileinput', 'fnmatch', 'formatter', 'fractions', 'ftplib', 'functools',
+ 'gc', 'getopt', 'getpass', 'gettext', 'glob', 'gzip', 'hashlib', 'heapq', 'hmac', 'html', 'http',
+ 'imaplib', 'imghdr', 'imp', 'importlib', 'inspect', 'io', 'ipaddress', 'itertools', 'json',
+ 'keyword', 'lib2to3', 'linecache', 'locale', 'logging', 'lzma', 'mailbox', 'mailcap', 'marshal',
+ 'math', 'mimetypes', 'modulefinder', 'multiprocessing', 'netrc', 'nntplib', 'numbers', 'operator',
+ 'optparse', 'os', 'ossaudiodev', 'parser', 'pathlib', 'pdb', 'pickle', 'pickletools', 'pipes',
+ 'pkgutil', 'platform', 'plistlib', 'poplib', 'posix', 'pprint', 'profile', 'pstats', 'pty',
+ 'pwd', 'py_compile', 'pyclbr', 'pydoc', 'queue', 'quopri', 'random', 're', 'readline',
+ 'reprlib', 'resource', 'rlcompleter', 'runpy', 'sched', 'secrets', 'select', 'selectors', 'shelve',
+ 'shlex', 'shutil', 'signal', 'site', 'smtpd', 'smtplib', 'sndhdr', 'socket', 'socketserver',
+ 'spwd', 'sqlite3', 'ssl', 'stat', 'statistics', 'string', 'stringprep', 'struct', 'subprocess',
+ 'sunau', 'symtable', 'sys', 'sysconfig', 'syslog', 'tabnanny', 'tarfile', 'telnetlib', 'tempfile',
+ 'termios', 'test', 'textwrap', 'threading', 'time', 'timeit', 'token', 'tokenize', 'trace',
+ 'traceback', 'tracemalloc', 'tty', 'turtle', 'types', 'typing', 'unicodedata', 'unittest',
+ 'urllib', 'uu', 'uuid', 'venv', 'warnings', 'wave', 'weakref', 'webbrowser', 'xdrlib', 'xml',
+ 'xmlrpc', 'zipapp', 'zipfile', 'zipimport', 'zlib'
+}
+
+# Known corrections for PyPI package names
+KNOWN_CORRECTIONS = {
+ 'dateutil': 'python-dateutil',
+ 'dotenv': 'python-dotenv',
+ 'docx': 'python-docx',
+ 'tesseract': 'pytesseract',
+ 'magic': 'python-magic',
+ 'multipart': 'python-multipart',
+ 'newspaper': 'newspaper3k',
+ 'srtm': 'elevation',
+ 'yaml': 'pyyaml',
+ 'zoneinfo': 'backports.zoneinfo'
+}
+
+# List of generic names to exclude
+EXCLUDED_NAMES = {'models', 'data', 'convert', 'example', 'tests'}
+
+def run_command(command: List[str]) -> Tuple[int, str, str]:
+ process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ stdout, stderr = process.communicate()
+ return process.returncode, stdout.decode(), stderr.decode()
+
+def is_package_installed(package: str) -> bool:
+ returncode, stdout, _ = run_command(["mamba", "list"])
+ if returncode == 0:
+ if re.search(f"^{package}\\s", stdout, re.MULTILINE):
+ return True
+ returncode, stdout, _ = run_command(["pip", "list"])
+ return re.search(f"^{package}\\s", stdout, re.MULTILINE) is not None
+
+def install_package(package: str):
+ if is_package_installed(package):
+ print(f"Package '{package}' is already installed.")
+ return
+
+ print(f"Installing package '{package}'.")
+ returncode, _, _ = run_command(["mamba", "install", "-y", "-c", "conda-forge", package])
+ if returncode != 0:
+ returncode, _, _ = run_command(["pip", "install", package])
+
+ if returncode != 0:
+ print(f"Failed to install package '{package}'.")
+ else:
+ print(f"Successfully installed package '{package}'.")
+
+def process_python_file(file_path: str) -> Set[str]:
+ with open(file_path, 'r') as file:
+ content = file.read()
+
+ imports = set()
+ for line in content.split('\n'):
+ line = line.strip()
+ if line.startswith(('import ', 'from ')) and not line.startswith('#'):
+ if line.startswith('import '):
+ modules = line.replace('import ', '').split(',')
+ for mod in modules:
+ mod = re.sub(r'\s+as\s+\w+', '', mod).split('.')[0].strip()
+ if mod and not mod.isupper() and mod not in EXCLUDED_NAMES and mod not in BUILTIN_MODULES:
+ imports.add(KNOWN_CORRECTIONS.get(mod, mod))
+ elif line.startswith('from '):
+ mod = line.split(' ')[1].split('.')[0].strip()
+ if mod and not mod.isupper() and mod not in EXCLUDED_NAMES and mod not in BUILTIN_MODULES:
+ imports.add(KNOWN_CORRECTIONS.get(mod, mod))
+
+ return imports
+
+def process_requirements_file(file_path: str) -> Set[str]:
+ with open(file_path, 'r') as file:
+ return {line.strip() for line in file if line.strip() and not line.startswith('#')}
+
+def main():
+ if len(sys.argv) < 2:
+ print("Usage: kip <package1> [<package2> ...] or kip <script.py> or kip -r <requirements.txt>")
+ sys.exit(1)
+
+ packages_to_install = set()
+
+ i = 1
+ while i < len(sys.argv):
+ arg = sys.argv[i]
+ if arg == '-r':
+ if i + 1 < len(sys.argv):
+ requirements_file = sys.argv[i + 1]
+ if os.path.isfile(requirements_file):
+ packages_to_install.update(process_requirements_file(requirements_file))
+ else:
+ print(f"Requirements file {requirements_file} not found.")
+ sys.exit(1)
+ i += 2
+ else:
+ print("Error: -r flag requires a file path.")
+ sys.exit(1)
+ elif arg.endswith('.py'):
+ if os.path.isfile(arg):
+ packages_to_install.update(process_python_file(arg))
+ else:
+ print(f"File {arg} not found.")
+ sys.exit(1)
+ i += 1
+ else:
+ packages_to_install.add(arg)
+ i += 1
+
+ for package in packages_to_install:
+ install_package(package)
+
+if __name__ == "__main__":
+ main()
diff --git a/linecount b/linecount
new file mode 100755
index 0000000..a689595
--- /dev/null
+++ b/linecount
@@ -0,0 +1,70 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+
+def is_binary(file_path):
+ """
+ Determines if a file is binary by checking its content.
+ Returns True for binary files, False for text files.
+ """
+ try:
+ with open(file_path, 'rb') as f:
+ # Read the first 1024 bytes to check if it's binary
+ chunk = f.read(1024)
+ if b'\0' in chunk:
+ return True
+ return False
+ except Exception as e:
+ print(f"Error reading file {file_path}: {e}")
+ return True # Treat unreadable files as binary for safety.
+
+def count_lines_in_file(file_path):
+ """
+ Counts the number of lines in a given text file.
+ """
+ try:
+ with open(file_path, 'r', encoding='utf-8', errors='ignore') as f:
+ return sum(1 for _ in f)
+ except Exception as e:
+ print(f"Error counting lines in file {file_path}: {e}")
+ return 0
+
+def count_lines_in_directory(directory, extensions=None):
+ """
+ Recursively counts lines in all text files (optionally filtered by extensions) within the directory.
+ """
+ total_lines = 0
+ total_files = 0
+ for root, _, files in os.walk(directory):
+ for file_name in files:
+ file_path = os.path.join(root, file_name)
+
+ # Skip binary files
+ if is_binary(file_path):
+ continue
+
+ # Check for extensions if provided
+ if extensions and not file_name.lower().endswith(tuple(extensions)):
+ continue
+
+ # Count lines in the valid file
+ lines = count_lines_in_file(file_path)
+ total_lines += lines
+ total_files += 1
+
+ return total_files, total_lines
+
+if __name__ == "__main__":
+ # Get extensions from command-line arguments
+ extensions = [ext.lower() for ext in sys.argv[1:]] if len(sys.argv) > 1 else None
+
+ # Get the current working directory
+ current_dir = os.getcwd()
+ print(f"Scanning directory: {current_dir}")
+ if extensions:
+ print(f"Filtering by extensions: {', '.join(extensions)}")
+ total_files, total_lines = count_lines_in_directory(current_dir, extensions)
+ print(f"Total matching files: {total_files}")
+ print(f"Total lines across matching files: {total_lines}")
+
diff --git a/lsd b/lsd
new file mode 100755
index 0000000..963f539
--- /dev/null
+++ b/lsd
@@ -0,0 +1,19 @@
+#!/bin/bash
+
+# Default options for lsd
+default_options="--color=always -F --long --size=short --permission=octal --group-dirs=first -X"
+
+# Check if the first argument is a directory or an option
+if [[ $# -gt 0 && ! $1 =~ ^- ]]; then
+ # First argument is a directory, store it and remove from arguments list
+ directory=$1
+ shift
+else
+ # No directory specified, default to the current directory
+ directory="."
+fi
+
+# Execute lsd with the default options, directory, and any additional arguments provided
+/opt/homebrew/bin/lsd $default_options "$directory" "$@"
+
+
diff --git a/mamba_exporter b/mamba_exporter
new file mode 100755
index 0000000..176e713
--- /dev/null
+++ b/mamba_exporter
@@ -0,0 +1,15 @@
+#!/bin/bash
+
+# List all conda environments and cut the output to get just the names
+envs=$(mamba env list | awk '{print $1}' | grep -v '^#' | grep -v 'base')
+
+# Loop through each environment name
+for env in $envs; do
+ # Use conda (or mamba, but conda is preferred for compatibility reasons) to export the environment to a YAML file
+ # No need to activate the environment; conda can export directly by specifying the name
+ echo "Exporting $env..."
+ mamba env export --name $env > "${env}.yml"
+done
+
+echo "All environments have been exported."
+
diff --git a/mamba_importer b/mamba_importer
new file mode 100755
index 0000000..79886e1
--- /dev/null
+++ b/mamba_importer
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+# Function to process a single .yml file
+process_file() {
+ file="$1"
+ if [[ -f "$file" ]]; then
+ env_name=$(echo "$file" | sed 's/.yml$//')
+ echo "Creating environment from $file..."
+ conda env create -f "$file" || echo "Failed to create environment from $file"
+ else
+ echo "File $file does not exist."
+ fi
+}
+
+# Check if a .yml file was provided as an argument
+if [[ $# -eq 1 && $1 == *.yml ]]; then
+ # Process the provided .yml file
+ process_file "$1"
+else
+ # No argument provided, process all .yml files in the current directory
+ for file in *.yml; do
+ process_file "$file"
+ done
+ echo "Environment creation process completed."
+fi
+
diff --git a/murder b/murder
new file mode 100755
index 0000000..3c76f1b
--- /dev/null
+++ b/murder
@@ -0,0 +1,24 @@
+#!/bin/bash
+
+# Check if an argument is given
+if [ $# -eq 0 ]; then
+ echo "Usage: murder [process name or port]"
+ exit 1
+fi
+
+# Get the input parameter
+ARGUMENT=$1
+
+# Check if the argument is numeric
+if [[ $ARGUMENT =~ ^[0-9]+$ ]]; then
+ echo "Killing processes listening on port $ARGUMENT"
+ lsof -t -i:$ARGUMENT | xargs kill
+else
+ # Process name was given instead of a port number
+ echo "Killing processes with name $ARGUMENT"
+ for PID in $(ps aux | grep $ARGUMENT | grep -v grep | awk '{print $2}'); do
+ echo "Killing process $PID"
+ sudo kill -9 $PID
+ done
+fi
+
diff --git a/n3k b/n3k
new file mode 100755
index 0000000..bb1785b
--- /dev/null
+++ b/n3k
@@ -0,0 +1,101 @@
+#!/usr/bin/env python3
+
+import sys
+import asyncio
+import trafilatura
+from newspaper import Article
+from urllib.parse import urlparse
+from datetime import datetime
+import math
+from typing import Optional
+import textwrap
+
+async def fetch_and_parse_article(url: str):
+ # Try trafilatura first
+ source = trafilatura.fetch_url(url)
+
+ if source:
+ try:
+ traf = trafilatura.extract_metadata(filecontent=source, default_url=url)
+
+ article = Article(url)
+ article.set_html(source)
+ article.parse()
+
+ # Update article properties with trafilatura data
+ article.title = article.title or traf.title or url
+ article.authors = article.authors or (traf.author if isinstance(traf.author, list) else [traf.author])
+ article.publish_date = traf.date or datetime.now()
+ article.text = trafilatura.extract(source, output_format="markdown", include_comments=False) or article.text
+ article.top_image = article.top_image or traf.image
+ article.source_url = traf.sitename or urlparse(url).netloc.replace('www.', '').title()
+
+ return article
+ except Exception:
+ pass
+
+ # Fallback to newspaper3k
+ try:
+ headers = {
+ 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36',
+ 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
+ }
+
+ article = Article(url)
+ article.config.browser_user_agent = headers['User-Agent']
+ article.config.headers = headers
+ article.download()
+ article.parse()
+
+ article.source_url = urlparse(url).netloc.replace('www.', '').title()
+ return article
+
+ except Exception as e:
+ raise Exception(f"Failed to parse article from {url}: {str(e)}")
+
+def format_article_markdown(article) -> str:
+ # Format title
+ output = f"# {article.title}\n\n"
+
+ # Format metadata
+ if article.authors:
+ authors = article.authors if isinstance(article.authors, list) else [article.authors]
+ output += f"*By {', '.join(filter(None, authors))}*\n\n"
+
+ if article.publish_date:
+ date_str = article.publish_date.strftime("%Y-%m-%d") if isinstance(article.publish_date, datetime) else str(article.publish_date)
+ output += f"*Published: {date_str}*\n\n"
+
+ if article.top_image:
+ output += f"\n\n"
+
+ # Format article text with proper wrapping
+ if article.text:
+ paragraphs = article.text.split('\n')
+ wrapped_paragraphs = []
+
+ for paragraph in paragraphs:
+ if paragraph.strip():
+ wrapped = textwrap.fill(paragraph.strip(), width=80)
+ wrapped_paragraphs.append(wrapped)
+
+ output += '\n\n'.join(wrapped_paragraphs)
+
+ return output
+
+async def main():
+ if len(sys.argv) != 2:
+ print("Usage: ./n3k <article_url>")
+ sys.exit(1)
+
+ url = sys.argv[1]
+ try:
+ article = await fetch_and_parse_article(url)
+ formatted_content = format_article_markdown(article)
+ print(formatted_content)
+ except Exception as e:
+ print(f"Error processing article: {str(e)}")
+ sys.exit(1)
+
+if __name__ == "__main__":
+ asyncio.run(main())
diff --git a/nocomment b/nocomment
new file mode 100755
index 0000000..2cb655e
--- /dev/null
+++ b/nocomment
@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+
+import sys
+import os
+
+def print_significant_lines(file_path):
+ try:
+ with open(file_path, 'r') as file:
+ for line in file:
+ # Strip whitespace from the beginning and end of the line
+ stripped_line = line.strip()
+
+ # Check if the line is not empty, not whitespace, and not a comment
+ if stripped_line and not stripped_line.startswith('#'):
+ print(line.rstrip()) # Print the line without trailing newline
+ except FileNotFoundError:
+ print(f"Error: File '{file_path}' not found.", file=sys.stderr)
+ except IOError:
+ print(f"Error: Unable to read file '{file_path}'.", file=sys.stderr)
+
+if __name__ == "__main__":
+ if len(sys.argv) != 2:
+ print("Usage: nocomment <file_path>", file=sys.stderr)
+ sys.exit(1)
+
+ file_path = sys.argv[1]
+ print_significant_lines(file_path)
diff --git a/noon b/noon
new file mode 100755
index 0000000..1c2bb0c
--- /dev/null
+++ b/noon
@@ -0,0 +1,116 @@
+#!/Users/sij/miniforge3/bin/python
+
+import os
+import sys
+import argparse
+
+def ask_for_confirmation(message):
+ while True:
+ user_input = input(message + " (y/n): ").strip().lower()
+ if user_input in ('y', 'n'):
+ return user_input == 'y'
+ else:
+ print("Invalid input. Please enter 'y' or 'n'.")
+
+def rename_files_orig(root_dir, manual):
+ for dirpath, _, filenames in os.walk(root_dir, followlinks=False):
+ for filename in filenames:
+ if '(orig)' in filename:
+ orig_filepath = os.path.join(dirpath, filename)
+ base_filename, ext = os.path.splitext(filename)
+ new_filename = base_filename.replace('(orig)', '')
+ new_filepath = os.path.join(dirpath, new_filename + ext)
+
+ if os.path.exists(new_filepath):
+ new_file_new_name = new_filename + '(new)' + ext
+ new_file_new_path = os.path.join(dirpath, new_file_new_name)
+
+ if manual:
+ if not ask_for_confirmation(f"Do you want to rename {new_filepath} to {new_file_new_path}?"):
+ continue
+
+ if os.path.exists(new_file_new_path):
+ print(f"Error: Cannot rename {new_filepath} to {new_file_new_path} because the target file already exists.")
+ continue
+
+ os.rename(new_filepath, new_file_new_path)
+ print(f'Renamed: {new_filepath} -> {new_file_new_name}')
+ else:
+ print(f"No associated file found for: {orig_filepath}")
+
+ orig_file_new_name = new_filename + ext
+ orig_file_new_path = os.path.join(dirpath, orig_file_new_name)
+
+ if manual:
+ if not ask_for_confirmation(f"Do you want to rename {orig_filepath} to {orig_file_new_path}?"):
+ continue
+
+ if os.path.exists(orig_file_new_path):
+ print(f"Error: Cannot rename {orig_filepath} to {orig_file_new_path} because the target file already exists.")
+ continue
+
+ os.rename(orig_filepath, orig_file_new_path)
+ print(f'Renamed: {orig_filepath} -> {orig_file_new_name}')
+
+def rename_files_new(root_dir, manual):
+ for dirpath, _, filenames in os.walk(root_dir, followlinks=False):
+ for filename in filenames:
+ if '(new)' in filename:
+ new_filepath = os.path.join(dirpath, filename)
+ base_filename, ext = os.path.splitext(filename)
+ orig_filename = base_filename.replace('(new)', '')
+ orig_filepath = os.path.join(dirpath, orig_filename + ext)
+
+ if os.path.exists(orig_filepath):
+ orig_file_orig_name = orig_filename + '(orig)' + ext
+ orig_file_orig_path = os.path.join(dirpath, orig_file_orig_name)
+
+ if manual:
+ if not ask_for_confirmation(f"Do you want to rename {orig_filepath} to {orig_file_orig_path}?"):
+ continue
+
+ if os.path.exists(orig_file_orig_path):
+ print(f"Error: Cannot rename {orig_filepath} to {orig_file_orig_path} because the target file already exists.")
+ continue
+
+ os.rename(orig_filepath, orig_file_orig_path)
+ print(f'Renamed: {orig_filepath} -> {orig_file_orig_name}')
+ else:
+ print(f"No associated file found for: {new_filepath}")
+
+ new_file_new_name = orig_filename + ext
+ new_file_new_path = os.path.join(dirpath, new_file_new_name)
+
+ if manual:
+ if not ask_for_confirmation(f"Do you want to rename {new_filepath} to {new_file_new_path}?"):
+ continue
+
+ if os.path.exists(new_file_new_path):
+ print(f"Error: Cannot rename {new_filepath} to {new_file_new_path} because the target file already exists.")
+ continue
+
+ os.rename(new_filepath, new_file_new_path)
+ print(f'Renamed: {new_filepath} -> {new_file_new_name}')
+
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser(description='Rename files based on given criteria.')
+ parser.add_argument('-o', '--orig', action='store_true', help='Rename files ending with (orig)')
+ parser.add_argument('-n', '--new', action='store_true', help='Rename files ending with (new)')
+ parser.add_argument('-m', '--manual', action='store_true', help='Manual mode: ask for confirmation before each renaming')
+ parser.add_argument('directory', nargs='?', default=os.getcwd(), help='Directory to start the search (default: current directory)')
+ args = parser.parse_args()
+
+ if args.orig and args.new:
+ print("Error: Please specify either -o or -n, not both.")
+ sys.exit(1)
+
+ if args.orig:
+ print("Running in ORIG mode")
+ rename_files_orig(args.directory, args.manual)
+ elif args.new:
+ print("Running in NEW mode")
+ rename_files_new(args.directory, args.manual)
+ else:
+ print("Error: Please specify either -o or -n.")
+ sys.exit(1)
+
diff --git a/nv b/nv
new file mode 100755
index 0000000..f0775e9
--- /dev/null
+++ b/nv
@@ -0,0 +1,40 @@
+#!/bin/bash
+
+SESSION_NAME="$1"
+PYTHON_VERSION_INPUT="${2:-3.10}" # Default to 3.8 if not specified
+
+# Normalize the Python version input
+if [[ "$PYTHON_VERSION_INPUT" =~ ^python[0-9.]+$ ]]; then
+ PYTHON_VERSION="${PYTHON_VERSION_INPUT//python/}"
+elif [[ "$PYTHON_VERSION_INPUT" =~ ^py[0-9.]+$ ]]; then
+ PYTHON_VERSION="${PYTHON_VERSION_INPUT//py/}"
+else
+ PYTHON_VERSION="$PYTHON_VERSION_INPUT"
+fi
+
+# Format for Conda
+MAMBA_PYTHON_VERSION="python=$PYTHON_VERSION"
+
+# Check if Conda environment exists
+if ! mamba env list | grep -q "^$SESSION_NAME\s"; then
+ echo "Creating new Mamba environment: $SESSION_NAME with $MAMBA_PYTHON_VERSION"
+ mamba create --name "$SESSION_NAME" "$MAMBA_PYTHON_VERSION" --yes
+fi
+
+# Find Conda env directory
+CONDA_ENV_DIR=$(mamba env list | grep "^$SESSION_NAME" | awk '{print $2}')
+
+# Handle tmux session
+if ! tmux has-session -t "$SESSION_NAME" 2>/dev/null; then
+ echo "Creating new tmux session: $SESSION_NAME"
+ tmux new-session -d -s "$SESSION_NAME"
+ sleep 2
+fi
+
+# Attach to tmux session and update PATH before activating Conda environment
+sleep 1
+tmux send-keys -t "$SESSION_NAME" "export PATH=\"$MAMBA_ENV_DIR/bin:\$PATH\"" C-m
+tmux send-keys -t "$SESSION_NAME" "source ~/.zshrc" C-m
+tmux send-keys -t "$SESSION_NAME" "mamba activate $SESSION_NAME" C-m
+tmux attach -t "$SESSION_NAME"
+
diff --git a/ocr b/ocr
new file mode 100755
index 0000000..937c042
--- /dev/null
+++ b/ocr
@@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+
+import sys
+import os
+from pathlib import Path
+from pdf2image import convert_from_path # This is the correct import
+import easyocr
+from PyPDF2 import PdfReader, PdfWriter
+import concurrent.futures
+import argparse
+from tqdm import tqdm
+import logging
+
+def setup_logging():
+ logging.basicConfig(
+ level=logging.INFO,
+ format='%(asctime)s - %(levelname)s - %(message)s',
+ handlers=[
+ logging.StreamHandler(),
+ logging.FileHandler('ocr_process.log')
+ ]
+ )
+
+def extract_images_from_pdf_chunk(pdf_path, start_page, num_pages):
+ try:
+ return convert_from_path(pdf_path, # This is the correct function name
+ first_page=start_page,
+ last_page=start_page + num_pages - 1,
+ dpi=300)
+ except Exception as e:
+ logging.error(f"Error extracting pages {start_page}-{start_page+num_pages}: {e}")
+ raise
+
+def process_page(image):
+ reader = easyocr.Reader(['en'], gpu=True)
+ return reader.readtext(image)
+
+def process_chunk(pdf_path, start_page, num_pages):
+ images = extract_images_from_pdf_chunk(pdf_path, start_page, num_pages)
+ results = []
+ with concurrent.futures.ThreadPoolExecutor() as executor:
+ futures = [executor.submit(process_page, image) for image in images]
+ for future in concurrent.futures.as_completed(futures):
+ try:
+ results.append(future.result())
+ except Exception as e:
+ logging.error(f"Error processing page: {e}")
+ return results
+
+def main():
+ parser = argparse.ArgumentParser(description='OCR a PDF file using EasyOCR')
+ parser.add_argument('pdf_path', type=str, help='Path to the PDF file')
+ parser.add_argument('--chunk-size', type=int, default=100,
+ help='Number of pages to process in each chunk')
+ args = parser.parse_args()
+
+ pdf_path = Path(args.pdf_path)
+ if not pdf_path.exists():
+ print(f"Error: File {pdf_path} does not exist")
+ sys.exit(1)
+
+ setup_logging()
+ logging.info(f"Starting OCR process for {pdf_path}")
+
+ # Create output directory
+ output_dir = pdf_path.parent / f"{pdf_path.stem}_ocr_results"
+ output_dir.mkdir(exist_ok=True)
+
+ reader = PdfReader(str(pdf_path))
+ total_pages = len(reader.pages)
+
+ with tqdm(total=total_pages) as pbar:
+ for start_page in range(1, total_pages + 1, args.chunk_size):
+ chunk_size = min(args.chunk_size, total_pages - start_page + 1)
+ chunk_output = output_dir / f"chunk_{start_page:06d}.txt"
+
+ if chunk_output.exists():
+ logging.info(f"Skipping existing chunk {start_page}")
+ pbar.update(chunk_size)
+ continue
+
+ try:
+ results = process_chunk(str(pdf_path), start_page, chunk_size)
+
+ # Save results
+ with open(chunk_output, 'w', encoding='utf-8') as f:
+ for page_num, page_results in enumerate(results, start_page):
+ f.write(f"=== Page {page_num} ===\n")
+ for text_result in page_results:
+ f.write(f"{text_result[1]}\n")
+ f.write("\n")
+
+ pbar.update(chunk_size)
+ logging.info(f"Completed chunk starting at page {start_page}")
+
+ except Exception as e:
+ logging.error(f"Failed to process chunk starting at page {start_page}: {e}")
+ continue
+
+ logging.info("OCR process complete")
+
+if __name__ == '__main__':
+ main()
+
diff --git a/ollapull b/ollapull
new file mode 100755
index 0000000..e77d776
--- /dev/null
+++ b/ollapull
@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# Pull the latest version of every locally installed model:
+
+ollama ls | tail -n +2 | awk '{print $1}' | while read MODEL; do
+ echo "Pulling latest for $MODEL..."
+ ollama pull "$MODEL"
+done
+
diff --git a/pf b/pf
new file mode 100755
index 0000000..ebc8ac0
--- /dev/null
+++ b/pf
@@ -0,0 +1,75 @@
+#!/usr/bin/python3
+import socket
+import threading
+import select
+
+def forward(source, destination):
+ try:
+ while True:
+ ready, _, _ = select.select([source], [], [], 1)
+ if ready:
+ data = source.recv(4096)
+ if not data:
+ break
+ destination.sendall(data)
+ except (OSError, socket.error) as e:
+ print(f"Connection error: {e}")
+ finally:
+ try:
+ source.shutdown(socket.SHUT_RD)
+ except OSError:
+ pass
+ try:
+ destination.shutdown(socket.SHUT_WR)
+ except OSError:
+ pass
+
+def handle(client_socket, remote_host, remote_port):
+ try:
+ remote_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ remote_socket.connect((remote_host, remote_port))
+
+ thread1 = threading.Thread(target=forward, args=(client_socket, remote_socket))
+ thread2 = threading.Thread(target=forward, args=(remote_socket, client_socket))
+
+ thread1.start()
+ thread2.start()
+
+ thread1.join()
+ thread2.join()
+ except Exception as e:
+ print(f"Error in handle: {e}")
+ finally:
+ client_socket.close()
+ remote_socket.close()
+
+def create_forwarder(local_host, local_port, remote_host, remote_port):
+ server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+ server_socket.bind((local_host, local_port))
+ server_socket.listen(5)
+
+ print(f"Forwarding {local_host}:{local_port} to {remote_host}:{remote_port}")
+
+ while True:
+ try:
+ client_socket, address = server_socket.accept()
+ print(f"Received connection from {address}")
+ threading.Thread(target=handle, args=(client_socket, remote_host, remote_port)).start()
+ except Exception as e:
+ print(f"Error accepting connection: {e}")
+
+def main():
+ listen_ip = '0.0.0.0'
+
+ imap_thread = threading.Thread(target=create_forwarder, args=(listen_ip, 1143, '127.0.0.1', 1142))
+ imap_thread.start()
+
+ smtp_thread = threading.Thread(target=create_forwarder, args=(listen_ip, 1025, '127.0.0.1', 1024))
+ smtp_thread.start()
+
+ imap_thread.join()
+ smtp_thread.join()
+
+if __name__ == "__main__":
+ main()
\ No newline at end of file
diff --git a/pippin b/pippin
new file mode 100755
index 0000000..9075300
--- /dev/null
+++ b/pippin
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# Check if an argument is provided
+if [ $# -eq 0 ]; then
+ echo "Usage: $0 <conda_environment_name>"
+ exit 1
+fi
+
+# Get the conda environment name from the command line argument
+env_name="$1"
+
+# Check if the conda environment already exists
+if ! conda info --envs | grep -q "^$env_name "; then
+ echo "Creating new conda environment: $env_name"
+ conda create -n "$env_name" python=3.9 -y
+else
+ echo "Conda environment '$env_name' already exists"
+fi
+
+# Activate the conda environment
+eval "$(conda shell.bash hook)"
+conda activate "$env_name"
+
+# Get the path to the conda environment's python binary
+conda_python=$(which python)
+
+# Recursively search for requirements.txt files and install dependencies
+find . -name "requirements.txt" | while read -r req_file; do
+ echo "Installing requirements from: $req_file"
+ "$conda_python" -m pip install -r "$req_file"
+done
+
+echo "All requirements.txt files processed."
+
diff --git a/pull b/pull
new file mode 100755
index 0000000..f50896d
--- /dev/null
+++ b/pull
@@ -0,0 +1,48 @@
+#!/bin/bash
+
+# Path to the file containing the list of repositories
+REPOS_FILE="$HOME/.repos.txt"
+
+# Check if the repos file exists
+if [ ! -f "$REPOS_FILE" ]; then
+ echo "Error: $REPOS_FILE does not exist in the current directory."
+ exit 1
+fi
+
+# Read the repos file and process each directory
+while IFS= read -r repo_path || [[ -n "$repo_path" ]]; do
+ # Trim whitespace
+ repo_path=$(echo "$repo_path" | xargs)
+
+ # Skip empty lines and lines starting with #
+ [[ -z "$repo_path" || "$repo_path" == \#* ]] && continue
+
+ # Expand tilde to home directory
+ repo_path="${repo_path/#\~/$HOME}"
+
+ echo "Processing repository: $repo_path"
+
+ # Navigate to the project directory
+ if ! cd "$repo_path"; then
+ echo "Error: Unable to change to directory $repo_path. Skipping."
+ continue
+ fi
+
+ # Check if it's a git repository
+ if [ ! -d .git ]; then
+ echo "Warning: $repo_path is not a git repository. Skipping."
+ continue
+ fi
+
+ # Force pull the latest changes from the repository
+ echo "Force pulling latest changes..."
+ git pull --force
+
+ # Return to the original directory
+ cd - > /dev/null
+
+ echo "Update complete for $repo_path"
+ echo "----------------------------------------"
+done < "$REPOS_FILE"
+
+echo "All repositories processed."
diff --git a/push b/push
new file mode 100755
index 0000000..237ec00
--- /dev/null
+++ b/push
@@ -0,0 +1,87 @@
+#!/bin/bash
+
+# Path to the file containing the list of repositories
+REPOS_FILE="$HOME/.repos.txt"
+
+# Check if the repos file exists
+if [ ! -f "$REPOS_FILE" ]; then
+ echo "Error: $REPOS_FILE does not exist."
+ exit 1
+fi
+
+# Read the repos file and process each directory
+while IFS= read -r repo_path || [[ -n "$repo_path" ]]; do
+ # Trim whitespace
+ repo_path=$(echo "$repo_path" | xargs)
+
+ # Skip empty lines and lines starting with #
+ [[ -z "$repo_path" || "$repo_path" == \#* ]] && continue
+
+ # Expand tilde to home directory
+ repo_path="${repo_path/#\~/$HOME}"
+
+ # Check if the directory exists
+ if [ ! -d "$repo_path" ]; then
+ echo "Warning: Directory $repo_path does not exist. Skipping."
+ continue
+ fi
+
+ echo "Processing repository: $repo_path"
+
+ # Navigate to the project directory
+ cd "$repo_path" || { echo "Error: Unable to change to directory $repo_path"; continue; }
+
+ # Check if it's a git repository
+ if [ ! -d .git ]; then
+ echo "Warning: $repo_path is not a git repository. Skipping."
+ continue
+ fi
+
+ # Check if 'origin' remote exists
+ if ! git remote | grep -q '^origin$'; then
+ echo "Remote 'origin' not found. Attempting to set it up..."
+ # Try to guess the remote URL based on the directory name
+ repo_name=$(basename "$repo_path")
+ remote_url="https://git.sij.ai/sij/$repo_name.git"
+ git remote add origin "$remote_url"
+ echo "Added remote 'origin' with URL: $remote_url"
+ fi
+
+ # Get the current branch
+ current_branch=$(git rev-parse --abbrev-ref HEAD)
+
+ # Pull the latest changes from the repository
+ echo "Pulling from $current_branch branch..."
+ if ! git pull origin "$current_branch"; then
+ echo "Failed to pull from origin. The remote branch might not exist or there might be conflicts."
+ echo "Skipping further operations for this repository."
+ continue
+ fi
+
+ # Add changes to the Git index (staging area)
+ echo "Adding all changes..."
+ git add .
+
+ # Check if there are changes to commit
+ if git diff-index --quiet HEAD --; then
+ echo "No changes to commit."
+ else
+ # Commit changes
+ echo "Committing changes..."
+ git commit -m "Auto-update: $(date)"
+
+ # Push changes to the remote repository
+ echo "Pushing all changes..."
+ if ! git push origin "$current_branch"; then
+ echo "Failed to push changes. The remote branch might not exist."
+ echo "Creating remote branch and pushing..."
+ git push -u origin "$current_branch"
+ fi
+ fi
+
+ echo "Update complete for $repo_path!"
+ echo "----------------------------------------"
+done < "$REPOS_FILE"
+
+echo "All repositories processed."
+
diff --git a/serv b/serv
new file mode 100755
index 0000000..d17d54c
--- /dev/null
+++ b/serv
@@ -0,0 +1,46 @@
+#!/bin/bash
+
+# Check if an executable is provided as an argument
+if [ -z "$1" ]; then
+ echo "Usage: $0 <executable>"
+ exit 1
+fi
+
+# Find the executable path using 'which'
+EXEC_PATH=$(which "$1")
+
+# Check if the executable exists
+if [ -z "$EXEC_PATH" ]; then
+ echo "Error: Executable '$1' not found."
+ exit 1
+fi
+
+# Get the executable name
+EXEC_NAME=$(basename "$EXEC_PATH")
+
+# Create the launchd plist file content
+PLIST_FILE_CONTENT="<?xml version=\"1.0\" encoding=\"UTF-8\"?>
+<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">
+<plist version=\"1.0\">
+<dict>
+ <key>Label</key>
+ <string>$EXEC_NAME</string>
+ <key>ProgramArguments</key>
+ <array>
+ <string>$EXEC_PATH</string>
+ </array>
+ <key>KeepAlive</key>
+ <true/>
+ <key>RunAtLoad</key>
+ <true/>
+</dict>
+</plist>"
+
+# Create the launchd plist file
+PLIST_FILE="$HOME/Library/LaunchAgents/$EXEC_NAME.plist"
+echo "$PLIST_FILE_CONTENT" > "$PLIST_FILE"
+
+# Load the launchd service
+launchctl load "$PLIST_FILE"
+
+echo "Service '$EXEC_NAME' has been created and loaded."
diff --git a/sij b/sij
new file mode 100755
index 0000000..88551bd
--- /dev/null
+++ b/sij
@@ -0,0 +1,23 @@
+#!/bin/bash
+
+# Set the path to the script
+SCRIPT_PATH="$HOME/workshop/sijapi/sijapi/helpers/start.py"
+
+# Check if the script exists
+if [ ! -f "$SCRIPT_PATH" ]; then
+ echo "Error: Script not found at $SCRIPT_PATH"
+ exit 1
+fi
+
+# Set up the environment
+source "$HOME/workshop/sijapi/sijapi/config/.env"
+
+# Activate the conda environment (adjust the path if necessary)
+source "$HOME/miniforge3/bin/activate" sijapi
+
+# Run the Python script with all command line arguments
+python "$SCRIPT_PATH" "$@"
+
+# Deactivate the conda environment
+conda deactivate
+
diff --git a/tablemd b/tablemd
new file mode 100755
index 0000000..c5c8fd4
--- /dev/null
+++ b/tablemd
@@ -0,0 +1,25 @@
+#!/usr/bin/env python3
+
+import sys
+import re
+
+def to_markdown_table(text):
+ lines = text.strip().split('\n')
+
+ # Using regex to split while preserving multi-word columns
+ pattern = r'\s{2,}' # Two or more spaces
+ rows = [re.split(pattern, line.strip()) for line in lines]
+
+ # Create the markdown header row
+ header = ' | '.join(rows[0])
+ # Create separator row with correct number of columns
+ separator = ' | '.join(['---'] * len(rows[0]))
+ # Create data rows
+ data_rows = [' | '.join(row) for row in rows[1:]]
+
+ # Combine all parts
+ return f"| {header} |\n| {separator} |\n" + \
+ '\n'.join(f"| {row} |" for row in data_rows)
+
+print(to_markdown_table(sys.stdin.read()))
+
diff --git a/tmux_merge b/tmux_merge
new file mode 100755
index 0000000..50ec168
--- /dev/null
+++ b/tmux_merge
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+# Get the first session as the target for all panes
+target_session=$(tmux list-sessions -F '#{session_name}' | head -n 1)
+target_window="${target_session}:0" # assuming the first window is index 0
+target_pane="${target_window}.0" # assuming the first pane is index 0
+
+# Loop through each session
+tmux list-sessions -F '#{session_name}' | while read session; do
+ # Skip the target session
+ if [[ "$session" == "$target_session" ]]; then
+ continue
+ fi
+
+ # Loop through each window in the session
+ tmux list-windows -t "$session" -F '#{window_index}' | while read window; do
+ # Loop through each pane in the window
+ tmux list-panes -t "${session}:${window}" -F '#{pane_index}' | while read pane; do
+ source="${session}:${window}.${pane}"
+ # Check if the source is not the same as the target
+ if [[ "$source" != "$target_pane" ]]; then
+ # Join the pane to the target pane
+ tmux join-pane -s "$source" -t "$target_pane"
+ fi
+ done
+ done
+
+ # After moving all panes from a session, kill the now-empty session
+ # Check if the session to be killed is not the target session
+ if [[ "$session" != "$target_session" ]]; then
+ tmux kill-session -t "$session"
+ fi
+done
+
+# After moving all panes, you may want to manually adjust the layout.
+# For a simple automatic layout adjustment, you can use:
+tmux select-layout -t "$target_window" tiled
+
+# Attach to the master session after everything is merged
+tmux attach-session -t "$target_session"
+
diff --git a/txt_line_merge_abc b/txt_line_merge_abc
new file mode 100755
index 0000000..d6487ea
--- /dev/null
+++ b/txt_line_merge_abc
@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+
+import sys
+
+def merge_files(file_paths):
+ if not file_paths:
+ print("At least one file path is required.")
+ return
+
+ # Read all lines from all files, including the first one
+ all_lines = set()
+ for file_path in file_paths:
+ with open(file_path, 'r') as f:
+ all_lines.update(f.read().splitlines())
+
+ # Sort the unique lines
+# sorted_lines = sorted(all_lines)
+ sorted_lines = sorted(all_lines, key=str.lower)
+
+
+ # Write the sorted, unique lines to the first file, overwriting its contents
+ with open(file_paths[0], 'w') as f:
+ for line in sorted_lines:
+ f.write(line + '\n')
+
+ print(f"Merged {len(file_paths)} files into {file_paths[0]}")
+
+if __name__ == "__main__":
+ # Get file paths from command line arguments
+ file_paths = sys.argv[1:]
+
+ if not file_paths:
+ print("Usage: txt-line-merge-abc file1.txt file2.txt file3.txt ...")
+ else:
+ merge_files(file_paths)
+
+
diff --git a/txtsort b/txtsort
new file mode 100755
index 0000000..a2235aa
--- /dev/null
+++ b/txtsort
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+# Checking if the user provided a file name
+if [ $# -ne 1 ]; then
+ echo "Usage: $0 filename"
+ exit 1
+fi
+
+# Checking if the given file is readable
+if ! [ -r "$1" ]; then
+ echo "The file '$1' is not readable or does not exist."
+ exit 1
+fi
+
+sort $1
+
diff --git a/uninstall b/uninstall
new file mode 100755
index 0000000..949f473
--- /dev/null
+++ b/uninstall
@@ -0,0 +1,118 @@
+#!/bin/bash
+
+# Required parameters:
+# @raycast.schemaVersion 1
+# @raycast.title Uninstall App
+# @raycast.mode fullOutput
+
+# Optional parameters:
+# @raycast.icon ποΈ
+# @raycast.argument1 { "type": "text", "placeholder": "App name" }
+
+# Documentation:
+# @raycast.description Move an application and its related files to the Trash (no interactive prompts)
+
+########################################
+# Moves a file to the Trash via AppleScript
+########################################
+move_to_trash() {
+ local file_path="$1"
+ osascript -e "tell application \"Finder\" to delete POSIX file \"$file_path\"" >/dev/null 2>&1
+}
+
+########################################
+# Uninstall the specified app name
+########################################
+uninstall_app() {
+ local input="$1"
+
+ # Ensure we have a .app extension
+ if [[ ! "$input" =~ \.app$ ]]; then
+ input="${input}.app"
+ fi
+
+ ########################################
+ # 1) Spotlight exact-match search
+ ########################################
+ local app_paths
+ app_paths=$(mdfind "kMDItemKind == 'Application' && kMDItemDisplayName == '$input'")
+
+ # 2) If nothing found, attempt partial-match on the base name (e.g. "Element")
+ if [ -z "$app_paths" ]; then
+ app_paths=$(mdfind "kMDItemKind == 'Application' && kMDItemDisplayName == '*${input%.*}*'")
+ fi
+
+ # 3) If still empty, bail out
+ if [ -z "$app_paths" ]; then
+ echo "Application not found. Please check the name and try again."
+ return 1
+ fi
+
+ ########################################
+ # Filter results to prefer /Applications
+ ########################################
+ # Turn multi-line results into an array
+ IFS=$'\n' read -rd '' -a all_matches <<< "$app_paths"
+
+ # We'll pick the match in /Applications if it exists.
+ local chosen=""
+ for path in "${all_matches[@]}"; do
+ if [[ "$path" == "/Applications/"* ]]; then
+ chosen="$path"
+ break
+ fi
+ done
+
+ # If no match was in /Applications, just pick the first one
+ if [ -z "$chosen" ]; then
+ chosen="${all_matches[0]}"
+ fi
+
+ # Show which one we're uninstalling
+ echo "Uninstalling: $chosen"
+
+ ########################################
+ # Move the .app bundle to Trash
+ ########################################
+ move_to_trash "$chosen"
+ echo "Moved $chosen to Trash."
+
+ ########################################
+ # Find bundle identifier for deeper cleanup
+ ########################################
+ local app_identifier
+ app_identifier=$(mdls -name kMDItemCFBundleIdentifier -r "$chosen")
+
+ echo "Removing related files..."
+
+ if [ -n "$app_identifier" ]; then
+ # Remove anything matching the bundle identifier
+ find /Library/Application\ Support \
+ /Library/Caches \
+ /Library/Preferences \
+ ~/Library/Application\ Support \
+ ~/Library/Caches \
+ ~/Library/Preferences \
+ -name "*$app_identifier*" -maxdepth 1 -print0 2>/dev/null \
+ | while IFS= read -r -d '' file; do
+ move_to_trash "$file"
+ done
+ else
+ # Fall back to removing by the app's base name
+ local base_name="${input%.app}"
+ find /Library/Application\ Support \
+ /Library/Caches \
+ /Library/Preferences \
+ ~/Library/Application\ Support \
+ ~/Library/Caches \
+ ~/Library/Preferences \
+ -name "*$base_name*" -maxdepth 1 -print0 2>/dev/null \
+ | while IFS= read -r -d '' file; do
+ move_to_trash "$file"
+ done
+ fi
+
+ echo "Uninstallation complete."
+}
+
+uninstall_app "$1"
diff --git a/vitals b/vitals
new file mode 100755
index 0000000..8ea3442
--- /dev/null
+++ b/vitals
@@ -0,0 +1,130 @@
+#!/bin/bash
+
+# Create a DNS rewrite rule in AdGuard home that assigns 'check.adguard.test'
+# to an IP address beginning '100.', such as the Tailscale IP of your server.
+# Alternatively, you can change the adguard_test_domain to whatever you like,
+# so long as it matches the domain of a DNS rewrite rule you created in AGH.
+
+adguard_test_domain='check.adguard.test'
+
+if [[ "$(uname)" == "Darwin" ]]; then
+ # macOS
+ local_ip=$(ifconfig | grep "inet " | grep -v 127.0.0.1 | awk '{print $2}' | head -n1)
+ uptime_seconds=$(sysctl -n kern.boottime | awk '{print $4}' | sed 's/,//')
+ current_time=$(date +%s)
+ uptime_seconds=$((current_time - uptime_seconds))
+ days=$((uptime_seconds / 86400))
+ hours=$(( (uptime_seconds % 86400) / 3600 ))
+ minutes=$(( (uptime_seconds % 3600) / 60 ))
+ uptime="up "
+ [[ $days -gt 0 ]] && uptime+="$days days, "
+ [[ $hours -gt 0 ]] && uptime+="$hours hours, "
+ uptime+="$minutes minutes"
+else
+ # Linux
+ local_ip=$(hostname -I | awk '{print $1}')
+ uptime=$(uptime -p)
+fi
+
+wan_info=$(curl -s --max-time 10 https://am.i.mullvad.net/json)
+wan_connected=false
+if [ ! -z "$wan_info" ]; then
+ wan_connected=true
+ wan_ip=$(echo "$wan_info" | jq -r '.ip')
+ mullvad_exit_ip=$(echo "$wan_info" | jq '.mullvad_exit_ip')
+ blacklisted=$(echo "$wan_info" | jq '.blacklisted.blacklisted')
+else
+ wan_ip="Unavailable"
+ mullvad_exit_ip=false
+ blacklisted=false
+fi
+
+# Check if Tailscale is installed and get IP
+if command -v tailscale &> /dev/null; then
+ has_tailscale=true
+ tailscale_ip=$(tailscale ip -4)
+ # Get Tailscale exit-node information
+ ts_exitnode_output=$(tailscale exit-node list)
+ # Parse exit node hostname
+ if echo "$ts_exitnode_output" | grep -q 'selected'; then
+ mullvad_exitnode=true
+ # Extract the hostname of the selected exit node, taking only the part before any newline
+ mullvad_hostname=$(echo "$ts_exitnode_output" | grep 'selected' | awk '{print $2}' | awk -F'\n' '{print $1}')
+ else
+ mullvad_exitnode=false
+ mullvad_hostname=""
+ fi
+else
+ has_tailscale=false
+ tailscale_ip="Not installed"
+ mullvad_exitnode=false
+ mullvad_hostname=""
+fi
+
+nextdns_info=$(curl -sL --max-time 10 https://test.nextdns.io)
+if [ -z "$nextdns_info" ]; then
+ echo "Failed to fetch NextDNS status or no internet connection." >&2
+ nextdns_connected=false
+ nextdns_protocol=""
+ nextdns_client=""
+else
+ nextdns_status=$(echo "$nextdns_info" | jq -r '.status')
+ if [ "$nextdns_status" = "ok" ]; then
+ nextdns_connected=true
+ nextdns_protocol=$(echo "$nextdns_info" | jq -r '.protocol')
+ nextdns_client=$(echo "$nextdns_info" | jq -r '.clientName')
+ else
+ nextdns_connected=false
+ nextdns_protocol=""
+ nextdns_client=""
+ fi
+fi
+
+# Check AdGuard Home DNS
+resolved_ip=$(dig +short $adguard_test_domain)
+if [[ $resolved_ip =~ ^100\. ]]; then
+ adguard_connected=true
+ adguard_protocol="AdGuard Home"
+ adguard_client="$resolved_ip"
+else
+ adguard_connected=false
+ adguard_protocol=""
+ adguard_client=""
+fi
+
+# Output JSON using jq for proper formatting and escaping
+jq -n \
+--arg local_ip "$local_ip" \
+--argjson wan_connected "$wan_connected" \
+--arg wan_ip "$wan_ip" \
+--argjson has_tailscale "$has_tailscale" \
+--arg tailscale_ip "$tailscale_ip" \
+--argjson mullvad_exitnode "$mullvad_exitnode" \
+--arg mullvad_hostname "$mullvad_hostname" \
+--argjson mullvad_exit_ip "$mullvad_exit_ip" \
+--argjson blacklisted "$blacklisted" \
+--argjson nextdns_connected "$nextdns_connected" \
+--arg nextdns_protocol "$nextdns_protocol" \
+--arg nextdns_client "$nextdns_client" \
+--argjson adguard_connected "$adguard_connected" \
+--arg adguard_protocol "$adguard_protocol" \
+--arg adguard_client "$adguard_client" \
+--arg uptime "$uptime" \
+'{
+ local_ip: $local_ip,
+ wan_connected: $wan_connected,
+ wan_ip: $wan_ip,
+ has_tailscale: $has_tailscale,
+ tailscale_ip: $tailscale_ip,
+ mullvad_exitnode: $mullvad_exitnode,
+ mullvad_hostname: $mullvad_hostname,
+ mullvad_exit_ip: $mullvad_exit_ip,
+ blacklisted: $blacklisted,
+ nextdns_connected: $nextdns_connected,
+ nextdns_protocol: $nextdns_protocol,
+ nextdns_client: $nextdns_client,
+ adguard_connected: $adguard_connected,
+ adguard_protocol: $adguard_protocol,
+ adguard_client: $adguard_client,
+ uptime: $uptime
+}'
diff --git a/vpn b/vpn
new file mode 100755
index 0000000..f4fa222
--- /dev/null
+++ b/vpn
@@ -0,0 +1,456 @@
+#!/usr/bin/env python3
+
+import subprocess
+import requests
+import argparse
+import json
+import random
+import datetime
+import os
+
+LOG_FILE = '/var/log/vpn_rotation.txt'
+
+PRIVACY_FRIENDLY_COUNTRIES = [
+ 'Finland',
+ 'Germany',
+ 'Iceland',
+ 'Netherlands',
+ 'Norway',
+ 'Sweden',
+ 'Switzerland'
+]
+
+TAILSCALE_ARGS = [
+ '--exit-node-allow-lan-access',
+ '--accept-dns',
+]
+
+def get_mullvad_info():
+ """Fetch JSON info from Mullvad's 'am.i.mullvad.net/json' endpoint."""
+ response = requests.get('https://am.i.mullvad.net/json')
+ if response.status_code != 200:
+ raise Exception("Could not fetch Mullvad info.")
+ return response.json()
+
+def get_current_exit_node():
+ """
+ Return the DNSName (e.g. 'de-ber-wg-001.mullvad.ts.net.') of whichever
+ peer is currently acting as the exit node. Otherwise returns None.
+ """
+ result = subprocess.run(['tailscale', 'status', '--json'],
+ capture_output=True, text=True)
+ if result.returncode != 0:
+ raise Exception("Failed to get Tailscale status")
+
+ status = json.loads(result.stdout)
+
+ # 'Peer' is a dict with keys like "nodekey:fe8efdbab7c2..."
+ peers = status.get('Peer', {})
+ for peer_key, peer_data in peers.items():
+ # If the node is currently the exit node, it should have "ExitNode": true
+ if peer_data.get('ExitNode') is True:
+ # Tailscale might return 'de-ber-wg-001.mullvad.ts.net.' with a trailing dot
+ dns_name = peer_data.get('DNSName', '')
+ dns_name = dns_name.rstrip('.') # remove trailing dot
+ return dns_name
+
+ # If we don't find any peer with ExitNode = true, there's no exit node
+ return None
+
+def list_exit_nodes():
+ """
+ Return a dict {node_name: country} of all available Tailscale exit nodes
+ based on 'tailscale exit-node list'.
+ The output lines typically look like:
+ <Star> <Name> <Country> <OS> ...
+ Example line:
+ * de-dus-wg-001.mullvad.ts.net Germany linux ...
+ """
+ result = subprocess.run(['tailscale', 'exit-node', 'list'], capture_output=True, text=True)
+ if result.returncode != 0:
+ raise Exception("Failed to list Tailscale exit nodes")
+
+ exit_nodes = {}
+ for line in result.stdout.splitlines():
+ parts = line.split()
+ # Basic sanity check for lines that actually contain node info
+ if len(parts) > 3:
+ # parts[0] might be "*" if it's the current node
+ # parts[1] is typically the FQDN (like "de-dus-wg-001.mullvad.ts.net")
+ # parts[2] is the Country
+ node_name = parts[1].strip()
+ node_country = parts[2].strip()
+ exit_nodes[node_name] = node_country
+
+ return exit_nodes
+
+def write_log(
+ old_node=None, new_node=None,
+ old_ip=None, new_ip=None,
+ old_country=None, new_country=None
+):
+ """
+ Appends a line to the log file reflecting a connection change.
+ Example:
+ 2025.01.17 01:11:33 UTC Β· disconnected from de-dus-wg-001.mullvad.ts.net (Germany)
+ Β· connected to at-vie-wg-001.mullvad.ts.net (Austria)
+ Β· changed IP from 65.21.99.202 to 185.213.155.74
+ If no old_node is specified, it indicates a fresh start (no disconnection).
+ If no new_node is specified, it indicates a stop (only disconnection).
+ """
+
+ utc_time = datetime.datetime.utcnow().strftime('%Y.%m.%d %H:%M:%S UTC')
+ log_parts = [utc_time]
+
+ # If old_node was present, mention disconnect
+ if old_node and old_country:
+ log_parts.append(f"disconnected from {old_node} ({old_country})")
+
+ # If new_node is present, mention connect
+ if new_node and new_country:
+ log_parts.append(f"connected to {new_node} ({new_country})")
+
+ # If IPs changed
+ if old_ip and new_ip and old_ip != new_ip:
+ log_parts.append(f"changed IP from {old_ip} to {new_ip}")
+
+ line = " Β· ".join(log_parts)
+
+ # Append to file
+ with open(LOG_FILE, 'a') as f:
+ f.write(line + "\n")
+
+def get_connection_history():
+ """
+ Returns an in-memory list of parsed log lines.
+ Each item looks like:
+ {
+ 'timestamp': datetime_object,
+ 'disconnected_node': '...',
+ 'disconnected_country': '...',
+ 'connected_node': '...',
+ 'connected_country': '...',
+ 'old_ip': '...',
+ 'new_ip': '...'
+ }
+ """
+ entries = []
+ if not os.path.isfile(LOG_FILE):
+ return entries
+
+ with open(LOG_FILE, 'r') as f:
+ lines = f.readlines()
+
+ for line in lines:
+ # Example line:
+ # 2025.01.17 01:11:33 UTC Β· disconnected from de-dus-wg-001.mullvad.ts.net (Germany) Β· connected to ...
+ # We'll parse step by step, mindful that each line can have different combos.
+ parts = line.strip().split(" Β· ")
+ if not parts:
+ continue
+
+ # parts[0] => '2025.01.17 01:11:33 UTC'
+ timestamp_str = parts[0]
+ connected_node = None
+ connected_country = None
+ disconnected_node = None
+ disconnected_country = None
+ old_ip = None
+ new_ip = None
+
+ # We parse the timestamp. We have '%Y.%m.%d %H:%M:%S UTC'
+ try:
+ dt = datetime.datetime.strptime(timestamp_str, '%Y.%m.%d %H:%M:%S UTC')
+ except ValueError:
+ continue # If it doesn't parse, skip.
+
+ for p in parts[1:]:
+ p = p.strip()
+ if p.startswith("disconnected from"):
+ # e.g. "disconnected from de-dus-wg-001.mullvad.ts.net (Germany)"
+ # We can split on "("
+ disc_info = p.replace("disconnected from ", "")
+ if "(" in disc_info and disc_info.endswith(")"):
+ node = disc_info.split(" (")[0]
+ country = disc_info.split(" (")[1].replace(")", "")
+ disconnected_node = node
+ disconnected_country = country
+ elif p.startswith("connected to"):
+ # e.g. "connected to at-vie-wg-001.mullvad.ts.net (Austria)"
+ conn_info = p.replace("connected to ", "")
+ if "(" in conn_info and conn_info.endswith(")"):
+ node = conn_info.split(" (")[0]
+ country = conn_info.split(" (")[1].replace(")", "")
+ connected_node = node
+ connected_country = country
+ elif p.startswith("changed IP from"):
+ # e.g. "changed IP from 65.21.99.202 to 185.213.155.74"
+ # We'll split on spaces
+ # changed IP from 65.21.99.202 to 185.213.155.74
+ # index: 0 1 2 3 4
+ ip_parts = p.split()
+ if len(ip_parts) >= 5:
+ old_ip = ip_parts[3]
+ new_ip = ip_parts[5]
+
+ entries.append({
+ 'timestamp': dt,
+ 'disconnected_node': disconnected_node,
+ 'disconnected_country': disconnected_country,
+ 'connected_node': connected_node,
+ 'connected_country': connected_country,
+ 'old_ip': old_ip,
+ 'new_ip': new_ip
+ })
+
+ return entries
+
+def get_last_connection_entry():
+ """
+ Parse the log and return the last entry that actually
+ has a 'connected_node', which indicates a stable connection.
+ """
+ history = get_connection_history()
+ # Go in reverse chronological order
+ for entry in reversed(history):
+ if entry['connected_node']:
+ return entry
+ return None
+
+def set_exit_node(exit_node):
+ """
+ Generic helper to set Tailscale exit node to 'exit_node'.
+ Returns (old_ip, new_ip, old_node, new_node, old_country, new_country)
+ """
+ # Get old info for logging
+ old_info = get_mullvad_info()
+ old_ip = old_info.get('ip')
+ old_country = old_info.get('country')
+ old_node = get_current_exit_node() # might be None
+
+ cmd = ['tailscale', 'set', f'--exit-node={exit_node}'] + TAILSCALE_ARGS
+ subprocess.run(cmd, check=True)
+
+ # Verify the new node
+ new_info = get_mullvad_info()
+ new_ip = new_info.get('ip')
+ new_country = new_info.get('country')
+ new_node = exit_node
+
+ return old_ip, new_ip, old_node, new_node, old_country, new_country
+
+def unset_exit_node():
+ """
+ Unset Tailscale exit node.
+ """
+ # For logging, we still want old IP + new IP. The 'new' IP after unsetting might revert to local.
+ old_info = get_mullvad_info()
+ old_ip = old_info.get('ip')
+ old_country = old_info.get('country')
+ old_node = get_current_exit_node()
+
+ cmd = ['tailscale', 'set', '--exit-node='] + TAILSCALE_ARGS
+ subprocess.run(cmd, check=True)
+
+ # Now see if the IP changed
+ new_info = get_mullvad_info()
+ new_ip = new_info.get('ip')
+ new_country = new_info.get('country')
+ new_node = None
+
+ write_log(old_node, new_node, old_ip, new_ip, old_country, new_country)
+ print("Exit node unset successfully!")
+
+def start_exit_node():
+ """
+ Start the exit node if none is currently set.
+ Otherwise, report what is already set.
+ """
+ current_exit_node = get_current_exit_node()
+ if current_exit_node:
+ print(f"Already connected to exit node: {current_exit_node}")
+ else:
+ # Use the default "tailscale exit-node suggest" approach
+ result = subprocess.run(['tailscale', 'exit-node', 'suggest'], capture_output=True, text=True)
+ if result.returncode != 0:
+ raise Exception("Failed to run 'tailscale exit-node suggest'")
+
+ suggested = ''
+ for line in result.stdout.splitlines():
+ if 'Suggested exit node' in line:
+ suggested = line.split(': ')[1].strip()
+ break
+
+ if not suggested:
+ raise Exception("No suggested exit node found.")
+
+ (old_ip, new_ip,
+ old_node, new_node,
+ old_country, new_country) = set_exit_node(suggested)
+
+ # Log it
+ write_log(old_node, new_node, old_ip, new_ip, old_country, new_country)
+ print(f"Exit node set successfully to {new_node}")
+
+def set_random_privacy_friendly_exit_node():
+ """
+ Pick a random node from PRIVACY_FRIENDLY_COUNTRIES and set it.
+ """
+ # Filter exit nodes by known privacy-friendly countries
+ nodes = list_exit_nodes()
+ # nodes is dict {node_name: country}
+ pf_nodes = [n for n, c in nodes.items() if c in PRIVACY_FRIENDLY_COUNTRIES]
+
+ if not pf_nodes:
+ raise Exception("No privacy-friendly exit nodes available")
+
+ exit_node = random.choice(pf_nodes)
+ (old_ip, new_ip,
+ old_node, new_node,
+ old_country, new_country) = set_exit_node(exit_node)
+
+ # Log
+ write_log(old_node, new_node, old_ip, new_ip, old_country, new_country)
+ print(f"Selected random privacy-friendly exit node: {exit_node}")
+ print("Exit node set successfully!")
+
+def set_random_exit_node_in_country(country_input):
+ """
+ Pick a random node in the given (case-insensitive) country_input.
+ Then set the exit node to that node.
+ """
+ country_input_normalized = country_input.strip().lower()
+
+ all_nodes = list_exit_nodes()
+ # Filter nodes in the user-requested country
+ country_nodes = [
+ node_name for node_name, node_country in all_nodes.items()
+ if node_country.lower() == country_input_normalized
+ ]
+
+ if not country_nodes:
+ raise Exception(f"No exit nodes found in {country_input}.")
+
+ exit_node = random.choice(country_nodes)
+
+ (old_ip, new_ip,
+ old_node, new_node,
+ old_country, new_country) = set_exit_node(exit_node)
+
+ # Log
+ write_log(old_node, new_node, old_ip, new_ip, old_country, new_country)
+ print(f"Selected random exit node in {country_input.title()}: {exit_node}")
+ print("Exit node set successfully!")
+
+def get_status():
+ """
+ Print current connection status:
+ - Whether connected or not
+ - Current exit node and IP
+ - Country of that exit node
+ - How long it has been connected to that exit node (based on the last log entry)
+ """
+ current_node = get_current_exit_node()
+ if not current_node:
+ print("No exit node is currently set.")
+ return
+
+ # Current IP & country
+ info = get_mullvad_info()
+ current_ip = info.get('ip')
+ current_country = info.get('country')
+
+ # Find the last time we connected to this node in the log
+ history = get_connection_history()
+ # We look from the end backwards for an entry that connected to the current_node
+ connected_since = None
+ for entry in reversed(history):
+ if entry['connected_node'] == current_node:
+ connected_since = entry['timestamp']
+ break
+
+ # We'll compute a "connected for X minutes/hours/days" style message
+ if connected_since:
+ now_utc = datetime.datetime.utcnow()
+ delta = now_utc - connected_since
+ # For user-friendliness, just show something like 1h 12m, or 2d 3h
+ # We'll do a simple approach:
+ total_seconds = int(delta.total_seconds())
+ days = total_seconds // 86400
+ hours = (total_seconds % 86400) // 3600
+ minutes = (total_seconds % 3600) // 60
+
+ duration_parts = []
+ if days > 0:
+ duration_parts.append(f"{days}d")
+ if hours > 0:
+ duration_parts.append(f"{hours}h")
+ if minutes > 0:
+ duration_parts.append(f"{minutes}m")
+ if not duration_parts:
+ duration_parts.append("0m") # means less than 1 minute
+
+ duration_str = " ".join(duration_parts)
+ print(f"Currently connected to: {current_node} ({current_country})")
+ print(f"IP: {current_ip}")
+ print(f"Connected for: {duration_str}")
+ else:
+ # If we never found it in the log, it's presumably a brand new connection
+ print(f"Currently connected to: {current_node} ({current_country})")
+ print(f"IP: {current_ip}")
+ print("Connected for: <unknown>, no log entry found.")
+
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser(description='Manage VPN exit nodes.')
+ parser.add_argument(
+ 'action',
+ choices=['start', 'stop', 'new', 'shh', 'to', 'status'],
+ help='Action to perform: start, stop, new, shh, to <country>, or status'
+ )
+ parser.add_argument(
+ 'country',
+ nargs='?',
+ default=None,
+ help='Country name (used only with "to" mode).'
+ )
+
+ args = parser.parse_args()
+
+ if args.action == 'start':
+ start_exit_node()
+ elif args.action == 'stop':
+ unset_exit_node()
+ elif args.action == 'new':
+ # This calls set_exit_node() using the Tailscale "suggest" approach
+ # from the original script
+ result = subprocess.run(['tailscale', 'exit-node', 'suggest'], capture_output=True, text=True)
+ if result.returncode != 0:
+ raise Exception("Failed to run 'tailscale exit-node suggest'")
+
+ exit_node = ''
+ for line in result.stdout.splitlines():
+ if 'Suggested exit node' in line:
+ exit_node = line.split(': ')[1].strip()
+ break
+
+ if not exit_node:
+ raise Exception("No suggested exit node found.")
+
+ (old_ip, new_ip,
+ old_node, new_node,
+ old_country, new_country) = set_exit_node(exit_node)
+ write_log(old_node, new_node, old_ip, new_ip, old_country, new_country)
+ print(f"Exit node set to suggested node: {new_node}")
+
+ elif args.action == 'shh':
+ # Random privacy-friendly
+ set_random_privacy_friendly_exit_node()
+
+ elif args.action == 'to':
+ # "vpn to sweden" => pick a random node in Sweden
+ if not args.country:
+ raise Exception("You must specify a country. e.g. vpn to sweden")
+ set_random_exit_node_in_country(args.country)
+
+ elif args.action == 'status':
+ get_status()
diff --git a/z b/z
new file mode 100755
index 0000000..62c3a7c
--- /dev/null
+++ b/z
@@ -0,0 +1,5 @@
+#!/bin/zsh
+source ~/.zshenv
+source ~/.zprofile
+source ~/.zshrc
+