Larry Law

Larry Law

Member of Technical Staff at DSO National Laboratories

Crash Course on using SoC Compute Clusters

1 minute read

Published:

Need access to NUS computing resources but not sure how? Here’s a quick crash course!

Logging In

# VPN
ssh larrylaw@xgpc2.comp.nus.edu.sg

# Otherwise through sunfire
ssh larrylaw@sunfire.comp.nus.edu.sg
ssh xgpc2
  1. To skip tunneling, either use ssh tunneling or SoC VPN.
  2. Compute cluster hardware configuration here.
  3. Use RSA key to skip typing of password. Guide here

Transfering Data

# From tembusu cluster to local 
## VPN
scp -r larrylaw@xgpc2.comp.nus.edu.sg:~/NM2/results/exp-e ./

## Otherwise through sunfire
scp -r larrylaw@sunfire.comp.nus.edu.sg:~/net_75 .
scp -r results/rs-obs/net_75/ larrylaw@sunfire.comp.nus
.edu.sg:~/

# From local to tembusu cluster
scp lab1.tar.gz larrylaw@xcne2.comp.nus.edu.sg:~/

Lazy to manually check cluster availability?

This bash script echos the availability of specified nodes.

#!/usr/bin/bash

echo "Checking all remote! /prays hard"

declare -a nodes=("xgpc" "xgpd" "xgpg")

rm output.txt

for ((i = 0; i < 10; i++)); do
    for node in "${nodes[@]}"
    do
        node_idx="${node}${i}"
        echo "$node_idx" >> output.txt
        echo yes | ssh -o ConnectTimeout=10 "larrylaw@$node_idx.comp.nus.edu.sg" nvidia-smi | grep "MiB /" >> output.txt
    done
done

echo "Go get em!"

Development

  1. pyenv for python version and pyvenv for virtual environment
  2. tmux to keep process running after ending ssh session. Help here.
  3. nvidia-smi to check GPU usage (before sending jobs)
  4. Remote development on VSCode. Help here.
  5. Speed up computation (significantly)) by storing data and outputs in /temp.
  6. View tensorboard on remote. Help here.
  7. Run on specific GPU via prepending CUDA_VISIBLE_DEVICES=2,3 python xxx.py

Comic

You May Also Enjoy

LLM Theory

less than 1 minute read

Published:

Awesome LLM theory articles I chanced upon

Simple tricks to improve Retrieval Augmented Generation (RAG) systems

3 minute read

Published:

Motivation

Retrieval Augmented Generation (RAG) is a framework for retrieving facts from an external knowledge base to ground large language models (LLMs) on the most accurate, up-to-date information. RAG is increasingly popular in industry as it’s simple to implement yet powerful. Here I’ll share some tricks to improve RAG systems.

Simple solution to training causal LMs with seq2seq objective

2 minute read

Published:

Motivation

How do we train causal language models (e.g. Alpaca, LLaMA, gpt-neox-20b…) with seq2seq objective? This goal is important because we want to instruction-tune our causal LMs, especially since Alpaca is the best open model at time of writing.