Albert Lea Data Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
<typo fs:xx-large; fw:bold;>Technical Information: Ubuntu Topics</typo> \\
[[tech:linux:ubuntu:dns|If you're setting up DNS services on Ubuntu server ...]]
====== Installation ======
===== Minimal Server =====
  * Install minimal server
    * Select "Entire disk with LVM" option
    * Select "Install OpenSSH Service" option
  * Perform updates
  * Show root partition with ''lvs'', extend if not already larger than 28G //(assuming the default root VG/LV naming)//:
    * ''lvextend -L +28G /dev/ubuntu-vg/ubuntu-lv''
    * ''resize2fs /dev/ubuntu-vg/ubuntu-lv''
  * If more than one disk is present, other disk(s) can be set up for LVM
    * Create LVM volume group on primary local storage:
      * ''pvcreate -M2 /dev/yourdiskdevice''
      * ''vgcreate -s 64 yourvgname /dev/yourdiskdevice''
  * Set up home LV, use either your default VG in case of single disk system, or new VG if other disk(s) available:
      * ''lvcreate -L 64G -n home yourvgname ; mkfs -t ext4 /dev/yourvgname/home''
      * Edit ''/etc/fstab'' and add entry to mount the new home LV on ''/home''
      * Move user home directories to new volume, with something like this: \\ //**NOTE**: best done as root, all other logins logged out//<code bash>cd /
mkdir home_
mv ./home/* ./home_/
mount /home
mv ./home_/* ./home/
rmdir home_</code>
  * Basic packages:
    * For wifi: wpasupplicant
    * For audio: jack2 pulseaudio pulseaudio-module-jack qjackctl
    * For X: xorg xinit xauth x11-xserver-utils xterm notion fvwm stalonetray
    * tigervnc-viewer
  * Networking ''/etc/netplan/whateveryourfilenameis.yaml'': \\ //(Note: ''wifis'' section optional)//<code yaml>network:
  version: 2
  renderer: networkd
  ethernets:
    enp0s31f6:
      dhcp4: true
      dhcp6: false
  wifis:
    wlp4s0:
      dhcp4: true
      dhcp6: false
      access-points:
        "YourESSID":
          password: "YourWifiPassword"</code>
    * For static addressing, add something like:<code yaml>    addresses: [192.168.0.105/24]
    gateway4: 192.168.0.9
    nameservers:
      addresses: [192.168.0.9]</code>

------
===== Virtualization Server =====
==== Bridged Networking ====
  * Packages: ''bridge-utils''
  * Populate ''/etc/sysctl.d/20-bridge.conf'' with:
    <code>net.bridge.bridge-nf-call-ip6tables=0
net.bridge.bridge-nf-call-iptables=0
net.bridge.bridge-nf-call-arptables=0</code>
  * Populate ''/etc/udev/rules.d/99-bridge.rules'' with:
    <code>ACTION=="add",SUBSYSTEM=="module",KERNEL=="br_netfilter",RUN+="/sbin/sysctl -p /etc/sysctl.d/20-bridge.conf"</code>
  * In ''/etc/netplan/whateveryourfilenameis.yaml'':<code yaml>network:
  version: 2
  renderer: networkd
  ethernets:
    enp4s0:
      dhcp4: false
      dhcp6: false
  bridges:
    br0:
      macaddress: DE:AD:BE:EF:10:01
      interfaces: [enp4s0]
      dhcp4: true
      parameters:
        stp: false
        forward-delay: 0</code>

==== LXC: Server Configuration ====
=== Limits ===
Official documentation: [[https://github.com/lxc/lxd/blob/master/doc/production-setup.md]]

In ''/etc/security/limits.conf'':
<code>
*                soft    nofile          1048576
*                hard    nofile          1048576
root             -       nofile          8192000
*                soft    memlock         unlimited
*                hard    memlock         unlimited
root             soft    memlock         unlimited
root             hard    memlock         unlimited
</code>

=== Kernel ===
In ''/etc/sysctl.conf'':
<code>
fs.aio-max-nr = 524288
fs.inotify.max_queued_events = 8192000
fs.inotify.max_user_instances = 8192000
fs.inotify.max_user_watches = 8192000
kernel.dmesg_restrict = 1
kernel.keys.maxbytes = 2000000
kernel.keys.maxkeys = 2048
net.core.bpf_jit_limit = 3000000000
net.ipv4.neigh.default.gc_thresh3 = 81920
net.ipv6.neigh.default.gc_thresh3 = 81920
vm.max_map_count = 262144
#net.ipv4.tcp_mem = 182757 243679 365514
net.core.netdev_max_backlog = 182757
</code>

=== Network ===
In ''/etc/udev/rules.d/90-net.rules'':
<code>
SUBSYSTEM=="net", ACTION=="add|change", KERNEL=="eth?" ATTR{tx_queue_len}="10000"
SUBSYSTEM=="net", ACTION=="add|change", KERNEL=="br?" ATTR{tx_queue_len}="10000"
</code>

------------
==== Basic Qemu ====
  * Packages: ''bridge-utils'', ''qemu-kvm''
  * Add bridge net config, in ''/etc/qemu/bridge.conf'':<code>allow br0</code>
  * Storage for configs + base images:
    * Create an LV + filesystem, and mount it at ''/var/lib/kvm''
    * Ownership should be ''root:disk'', with permissions ''2775'' //(drwxrwsr-x)//
  * LVs (LVM Logical Volumes) can be presented to a VM as a physical disk. \\ So, here is one recommended practice:
    * Perform a //"baseline install"// of an OS into a VM with a single virtual disk
      * Create baseline virtual disk: ''qemu-img create -f qcow2 baseline-vda 18G'' //(18G is decent for this purpose)//
      * Configure VM with a generic/baseline name reflecting the OS
      * Install OS and perform updates.  Use a generic login with generic password, and shut down VM
      * Copy the disk image to archive/repository location //(/var/lib/kvm/iso)//
        * Name the copy to reflect the OS, version, and date
        * Ownership ''root:disk'' and permissions ''444''
    * When creating a new VM using the baseline OS, create the new VM's virtual disk as a layer on top of the baseline image:<code>qemu-img create -f qcow2 -b /path/to/baseline/image vmname-vda 18G</code>
    * The VMs virtual disk is for the OS //(root filesystem)//, \\ create LVs to be presented to the VM as other disk(s):
      * Example: ''lvcreate -L 32G -n vmnamehome yourvgname''
      * Note: the sample start-up script below shows how the VM is presented with the LV as a physical disk
      * From inside the VM:
        * The physical disk will be seen as ''vdb'' or ''vdc'', etc (depending on how many)
        * Create filesystem directly on the physical device
        * Add entry to ''/etc/fstab'' to mount the filesystem at desired location
  * Sample VM start-up script: <code bash>
#!/bin/bash
VMN=yourvmname
VNC="12"
MAC0="DE:AD:BE:EF:00:12"

VMH=/var/lib/kvm
CPUT=EPYC
CPUS=4
VMM=8G
VGA="-vga virtio -display gtk"
# Headless ...
#VGA="-vnc localhost:${VNC} -daemonize"

VGN="yourvgname"
SIM=
HD0=
HD1=
HD2=
NIC0=
NIC1=
TPM=
SND=

# Boot device - d=cdrom, c=disk
BOOT="-boot c"

function initTPM {
	TPD=$(ps -ef | grep swtpm | grep -v grep)
	if [ "${TPM}" != "" -a "${TPD}" == "" ]; then
		swtpm socket -d --tpmstate dir=/var/lib/kvm/${VMN}/tpm --ctrl type=unixio,path=/var/lib/kvm/${VMN}/tpm/sock --log file=tpm.log,level=20 --tpm2
	else echo "NFO=>>> software TPM daemon detected, skipping its startup" ; fi
}

function vmOptionsWin11 {
	CPUT=EPYC,kvm_pv_eoi=on
#	SIM="-cdrom ${VMH}/iso/Win11_English_x64v1.iso"
#	SIM="-cdrom ${VMH}/iso/virtio-win-0.1.208.iso"
	HD0="-hda vda"
	HD1="-drive file=/dev/${VGN}/dwin,index=1,media=disk,format=raw"
#	NIC0="-netdev bridge,id=${VMN}eth0,br=br0 -device e1000,netdev=${VMN}eth0,mac=${MAC0}"
	NIC0="-netdev bridge,id=${VMN}eth0,br=br0 -device virtio-net-pci,netdev=${VMN}eth0,mac=${MAC0}"
	TPM="-bios /usr/share/ovmf/OVMF.fd -chardev socket,id=chrtpm,path=/var/lib/kvm/${VMN}/tpm/sock -tpmdev emulator,id=tpm0,chardev=chrtpm -device tpm-tis,tpmdev=tpm0"
	initTPM
}

function vmOptionsLinux {
#	SIM="-cdrom ${VMH}/iso/ubuntu-20.04.3-live-server-amd64.iso"
	HD0="-drive file=vda,if=virtio"
	HD1="-drive file=vdb,if=virtio"
	HD2="-drive file=/dev/${VGN}/${VMN}home,if=virtio,index=2,media=disk,format=raw"
	NIC0="-netdev bridge,id=${VMN}eth0,br=br0 -device virtio-net-pci,netdev=${VMN}eth0,mac=${MAC0}"
}

function vmOptions {
	vmOptionsLinux
#	vmOptionsWin11
}

cd ${VMH}/${VMN}
vmOptions
qemu-system-x86_64 -machine pc,accel=kvm -name "${VMN}" -cpu ${CPUT} -smp ${CPUS} -m ${VMM} ${BOOT} ${VGA} ${NIC0} ${NIC1} ${HD0} ${HD1} ${HD2} ${SIM} ${TPM} ${SND}
</code>

==== Advanced/Other Virtualization ====
  * Packages: ''zfsutils'', ''libvirt-clients'', ''libvirt-daemon-system'', ''virt-manager''
    * Add a bridge to libvirt:
      * Create a file ''/tmp/br0.xml'':<code xml><network>
  <name>br0</name>
  <forward mode='bridge'/>
  <bridge name='br0'/>
</network></code>
      * Add the new network via libvirt, and autostart it:<code bash>virsh net-define /tmp/br0.xml
virsh net-start br0
virsh net-autostart br0</code>
    * List VMs: ''virsh list --all''
    * Start VM: ''virsh start vmname''
    * Stop VM: ''virsh shutdown vmname''
  * Docker installation notes, see [[https://linuxconfig.org/how-to-install-docker-on-ubuntu-18-04-bionic-beaver]]
  * Docker dependencies: ''apt-transport-https'', ''ca-certificates'', ''software-properties-common''
  * In ''/etc/apt/sources.list.d/docker.list'':<code>deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable</code>
  * Add docker key: ''curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -''
  * ''apt update''
  * ''apt install docker-ce''

------
===== Basic configuration ... =====
  * In ''/etc/cloud/cloud.cfg'', change the ''preserve_hostname'' setting to ''true''
  * Set hostname - as root:
    * ''hostnamectl set-hostname //<newhostname>//''
    * ''hostnamectl set-hostname --static //<newhostname>//''
    * ''hostnamectl set-hostname --pretty //<newhostname>//''
    * ''hostnamectl set-hostname --transient //<newhostname>//''
  * Remove apparmor:
    * ''/etc/init.d/apparmor stop''
    * ''update-rc.d -f apparmor remove''

===== Extra packages =====
  * Primary 32-bit libraries: ''apt-get install libc6:i386 libstdc++6:i386''
  * vim-gtk
  * openssh-server
  * ksh
  * htop
  * git
  * lvm2 // ( ... for aes-xts-plain encryption support, among other things ... ) //
  * zfsutils-linux
  * qemu //(Optional: virtualbox)//
  * smbclient
  * cifs-utils
  * mysql-client
  * openjdk-8-jdk \\ //for openjdk-9-jre: ''apt -o Dpkg::Options::="--force-overwrite" install openjdk-9-jdk''//
  * tigervnc-viewer
  * tigervnc-standalone-server
  * Optional (audio): cpufrequtils dbus-x11 alsa jack pulseaudio-module-jack qjackctl mpg321 moc

===== Audio Configuration =====
==== Default Audio Device ====
This should be in ''/etc/security/limits.d/audio.conf'' or ''/etc/security/limits.conf'':<code>
@audio   -  rtprio     95
@audio   -  memlock    unlimited
</code>
Replace ''1'' with ALSA card number ''/etc/asound.conf'':<code>
defaults.pcm.card 1
defaults.ctl.card 1</code>

==== Disable PulseAudio auto-spawn ====
If you want to run Jack for low-latency high-performance audio, you'll need to make sure that Jack can grab your audio interface directly, which means you won't want PulseAudio to grab it on startup.  Disabling PulseAudio auto-spawn is most likely necessary, so you can start it manually after Jack is running:
  * Change in ''/etc/pulse/client.conf'': <code>autospawn = no
daemon-binary = /bin/true 
</code>
  * Rename startup file: <code bash>mv /etc/rc2.d/S50pulseaudio /etc/rc2.d/K50pulseaudio</code>

==== Route PulseAudio through Jack ====
On Debian-like systems, be sure to install ''pulseaudio-module-jack''.  Load with:
<code>
pacmd load-module module-native-protocol-unix
pacmd load-module module-jack-source channels=2
pacmd load-module module-jack-sink channels=2
pacmd set-default-sink jack_out
pacmd set-default-source jack_in
</code>
// (TODO: Add information about Fedora, OpenSuSE and the lot) // \\
Then, if you intend to run jackd all the time, in ''/etc/pulse/default.pa'' or ''~/.pulse/default.pa'':
<code>
load-module module-native-protocol-unix
load-module module-jack-sink channels=2
load-module module-jack-source channels=2
load-module module-null-sink
load-module module-stream-restore
load-module module-rescue-streams
load-module module-always-sink
load-module module-suspend-on-idle
set-default-sink jack_out
set-default-source jack_in
</code>

The following isn't strictly necessary but might be useful, in ''~/.pulse/daemon.conf'':
<code>
default-sample-format = float32le
default-sample-rate = 48000
realtime-scheduling = yes
exit-idle-time = -1
</code>

==== For mkusb: ====
As root:<code bash>add-apt-repository ppa:mkusb/ppa  # and press Enter
apt update
apt install mkusb</code>

==== 32-bit Compatibility ====
  * Basic activation: \\ ''dpkg --add-architecture i386'' \\ ''apt-get update''
  * Primary libraries: \\ ''apt-get install libc6:i386 libstdc++6:i386''
  * 32-bit pre-requisites for X32-Edit: <code>libx11-6:i386 libxext6:i386 libasound2:i386 libc6:i386 libfreetype6:i386 libc6:i386 libstdc++6:i386 libgcc1:i386 libxcb1:i386 zlib1g:i386 libpng12-0:i386 libxau6:i386 libxdmcp6:i386</code>

==== Skype for 16.04 ====
<code>
add-apt-repository "deb http://archive.canonical.com/ $(lsb_release -sc) partner"
apt-get update
apt-get install skype
</code>

===== MicroK8s Node =====
  * Create storage for Docker and mount at ''/var/lib/docker''
  * Run: ''snap install microk8s --classic''
  * Run: ''snap alias microk8s.kubectl kubectl''
  * Run: ''microk8s.enable dns storage ingress registry'' // **Optional:** dashboard //
  * To stop microk8s: ''microk8s.stop'' ... to start again: ''microk8s.start''
  * Storage volumes can be created/mounted and migrated to:
    * Kubernetes footprint storage is at: ''/var/snap/microk8s/common/var''
    * Persistent Volume Storage is at: ''/var/snap/microk8s/common/default-storage''
  * Credentials/Tokens
    * Run: ''kubectl -n kube-system get secret''
    * Identify your secret name and run: ''kubectl -n kube-system describe secret yoursecretname''
  * Show dashboard cluster IP address:
    * Run: ''kubectl get all --all-namespaces |grep kubernetes-dashboard | grep ClusterIP''
    * SSH tunnel to port ''443'' at that address, then point web browser at tunnelled port
  * Alternate approach:
    * Proxy for dashboard web access: \\ ''kubectl proxy --accept-hosts=.* --address=0.0.0.0 &''
    * [[http://k8s:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/]]

===== LXC Ubuntu-specifics =====
[[https://help.ubuntu.com/lts/serverguide/lxc.html]]
=== Basic init steps for Ubuntu ===
  * To enable passing block devices into LXC containers:
    * Add to ''config'' for container:
      * ''lxc.apparmor.profile = lxc-container-default-with-mounting''
    * Add in ''/etc/apparmor.d/lxc/lxc-default-with-mounting'':
      * ''mount options=(rw, bind, ro),''
    * And ... ''/etc/init.d/apparmor reload''
  * Use "download" template, like: \\ ''lxc-create -t download -n name -- -d ubuntu -r zesty -a amd64''
  * Edit config:
    * Set MAC address, if necessary
    * Ensure bridge is used (might have to switch from ''lxcbr0'' to ''br0'')
    * Add any mounts for container, like:
      * ''lxc.mount.entry = /dev/yourvg/yourlv dev/yourvg/yourlv none bind,create=file 0 0''
      * ''lxc.cgroup.devices.allow = b 253:13 rwm''
  * Attach to container for first fundamentals:
    * Add vital packages:  ''net-tools'' and ''openssh-server''
    * Create initial logins

==== Details to intialize for a new LXC container instance ====
  * Set power management, so machine won't sleep or turn off display
  * Disable floppy module<code bash>rmmod floppy
echo "blacklist floppy" > /etc/modprobe.d/blacklist-floppy.conf
dpkg-reconfigure initramfs-tools</code>

===== Activate 32-bit compatibility =====
  * ''dpkg --add-architecture i386''
  * ''apt-get update''

====== Distribution Trivia ======

==== Prevent Sleep/Hibernate ====
<code>systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target</code>

==== DNS Cache Purge ====
<code bash>systemd-resolve --flush-caches</code>

===== Gnome Desktop Topics =====
==== Reverting to console boot ====
  * Edit ''/etc/default/grub'':<code>GRUB_CMDLINE_LINUX_DEFAULT="text"
GRUB_TERMINAL=console</code>
  * Run:<code bash>update-grub
systemctl set-default multi-user.target</code>
  * Undoing text-mode:<code bash>systemctl set-default graphical.target</code>

==== Application Menu ====
=== Add an item ===
  * Create a file in ''~/.local/share/applications'' named ''yourapp.desktop'' with:<code>#!/usr/bin/env xdg-open
[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Terminal=false
Exec=/home/you/bin/yourpgm
Name=YourProgramName
Icon=your-icon-name</code>
  * Icons are typically under ''/usr/share/icons'' and can be either ''.png'' or ''.svg'' files

===== Upgrading =====
The ''/boot'' partition can fill up with old kernels unless periodically purged. \\
To see how many old kernels are present:
<code bash>apt list --installed | grep 'linux-image'</code>
There's little/no reason to keep more than one old kernel, so removing all but the most recent is advised. \\
After upgrading, if a new kernel is installed, keeping the most recent two is fine.

===== Disable IPv6 =====
Put into /etc/sysctl.conf:<code>
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
</code>
Run to re-load: ''sysctl -p''

===== Enable TCP listen for X =====
==== X TCP Listen in Gnome ====
In ''/etc/gdm3/custom.conf'' : <code>[security]
DisallowTCP=false

[xdmcp]
ServerArguments=-listen tcp</code>

==== X TCP Listen in XFCE ====
Populate ''/etc/lightdm/lightdm.conf'' with:<code>
[SeatDefaults]
greater-session=unity-greater
user-session=ubuntu
xserver-allow-tcp=true

[XDMCPServer]
enabled=true</code>

====== Some details for lighttpd ======
  * Installation: ''apt-get install lighttpd''
  * PHP, install: ''apt-get install php-cgi''
  * FastCGI, enable:  ''lighty-enable-mod fastcgi''
  * FastCGI for PHP, enable:  ''lighty-enable-mod fastcgi-php''
  * MySQL support, install: ''apt-get install php-mysql''
  * GD support, install: ''apt-get install php-gd''

====== Some Issues -> Solutions ======
==== Boot waiting for disconnected wired NIC ====
<code>
systemctl disable systemd-networkd-wait-online.service
systemctl mask systemd-networkd-wait-online.service
</code>
//(second line prevents the ''wait-online'' service from starting if requested by another service)//

==== Deja Dup backup in 16.04 (Unity) ====
  * Duplicity is not installed by default:  ''apt-get install duplicity''
  * Python module gi.repository needs to be "fixed":  ''apt-get install --reinstall python-gi''
==== Thunar (file manager) Network Browsing ====
  * Make sure smbclient and cifs-utils are installed
  * Add ''wins'' to end of hosts line in ''/etc/nsswitch.conf''
  * Just add ''cifs'' mounts for locations desired

==== Permission denied for apt ====
When installing a ''.deb'' package using ''apt'', and the package file is placed under the ''root'' user's home directory (typically ''/root''), permissions will usually not allow the ''_apt'' user to access the ''.deb'' file for everything it needs to. The error would show up something like this:<code>
Download is performed unsandboxed as root as file <somepathtofile> couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
</code>
In this case, the ''/root'' directory is probably set to ''drwx------'' for permissions, so changing it to allow the ''root'' group access fixes the problem:<code bash>chmod 775 /root</code>

====== Some Hardware Trivia ======

==== ACEPC W5 Pro: wifi ====
<code>cd /lib/firmware/brcm ; cp brcmfmac43455-sdio.raspberrypi,4-model-b.txt brcmfmac43455-sdio.txt</code>

==== Tune CPU frequency scaling for low-latency audio ====
<code bash>cpufreq-set -g performance</code>

==== Disable auto-sleep/auto-suspend ====
//(use ''unmask'' to re-enable)//
<code bash>systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target</code>

==== Disable screen blanking ====
... or, reactivate a deactivated display // (happens with some HDMI) //
<code bash>xset -dpms s off s noblank s 0 0 s noexpose</code>

==== EXFAT filesystem support ====
<code bash>apt install exfat-utils exfat-fuse</code>

==== ralink RT2790 issue(s) ====
This should resolve the issue:
<code bash>
modprobe -rv rt2800pci
modprobe -v rt2800pci nohwcrypt=Y
</code>

Put this in ''/etc/modprobe.d/rt2800pci.conf'' to make it permanent:
<code>options rt2800pci nohwcrypt=Y</code>

==== SecureBoot: Signed Kernel ====
[[https://blog.ubuntu.com/2017/08/11/how-to-sign-things-for-secure-boot]] \\

''openssl req -config ./openssl.conf -new -x509 -newkey rsa:2048 -nodes -days 36500 -outform DER -keyout "MOK.priv" -out "MOK.der"'' \\

''mokutil --import MOK.der'' \\

''mv vmlinuz-4.18.20-041820-generic vmlinuz-4.18.20-041820-generic-unsigned'' \\

''openssl x509 -in MOK.der -inform DER -outform PEM -out MOK.pem'' \\

''sbsign --key MOK.priv --cert MOK.pem /boot/vmlinuz-4.18.20-041820-generic-unsigned --output /boot/vmlinuz-4.18.20-041820-generic'' \\


<WRAP prewrap><code>
# This definition stops the following lines choking if HOME isn't
# defined.
HOME                    = .
RANDFILE                = $ENV::HOME/.rnd 
[ req ]
distinguished_name      = kernel_signing
x509_extensions         = v3
string_mask             = utf8only
prompt                  = no

[ kernel_signing ]
countryName             = US
stateOrProvinceName     = Minnesota
localityName            = Albert Lea
0.organizationName      = Albert Lea Data
commonName              = Secure Boot Signing
emailAddress            = kernelsigning@albertleadata.com

[ v3 ]
subjectKeyIdentifier    = hash
authorityKeyIdentifier  = keyid:always,issuer
basicConstraints        = critical,CA:FALSE
# Only include the second for module-signing
#extendedKeyUsage        = codeSigning,1.3.6.1.4.1.311.10.3.6,1.3.6.1.4.1.2312.16.1.2
extendedKeyUsage        = codeSigning,1.3.6.1.4.1.311.10.3.6
nsComment               = "OpenSSL Generated Certificate"
</code></WRAP>



 \\
// Links: [[:tech:linux:start|Linux Info]] ... [[:tech:start|Tech Info]] //
Albert Lea Data Wiki

User Tools

Site Tools

Page Tools
