https://github.com/agoose77/texat-local-cluster
Science Score: 13.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.5%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: agoose77
- Language: Shell
- Default Branch: main
- Size: 13.7 KB
Statistics
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
TexAT Local Cluster
This repository contains the necessary scripts to launch a dask cluster using local shared storage.
Usage
- Ensure that this directory is accessible on all necessary hosts
- Use rsync or a shared file system.
- Launch the scheduler on a machine using
launch-scheduler - Launch the worker(s) on each machine using
launch-scheduler- See script for environment variable arguments.
Requirements
texat-runtime.sifonPATHtexat-sim-datainshare/
Note
This repository uses direnv to set the PATH variable appropriately. If direnv is not available, use export PATH="$PWD/bin:$PWD/scripts:$PATH".
Architecture
In the bin directory, python3 and python point to the texat shell script. This launches texat-runtime.sif, which is a Singularity image built from the texat Dockerfile runtime target.
Singularity is used because it simplifies the commandline (shorter binding), runs as user (avoids UID mapping), is easily launched without importing, and is a requirement to launch on the SLURM cluster (therefore we use it here too).
The shared and backend directories are placeholders for the small and large file storage respectively. It is unlikely that these will be individually mapped instead of this entire directory being added to shared storage.
SSHFS
For local clusters, where bandwidth is large, we might disable compression:
bash
sshfs -o cache=yes -o compression=no -o reconnect,kernel_cache,large_read,allow_other,default_permissions,follow_symlinks <HOST>:/var/lib/cluster /mnt/cluster
Owner
- Name: Angus Hollands
- Login: agoose77
- Kind: user
- Location: United Kingdom
- Company: 2i2c
- Twitter: agoose77
- Repositories: 230
- Profile: https://github.com/agoose77
Open Source Infrastructure Engineer @ 2i2c. Executable Books core team member. PhD in Nuclear Physics from the University of Birmingham.
GitHub Events
Total
Last Year
Issues and Pull Requests
Last synced: over 1 year ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0