timbrado-${__time(yyyyMMddHHmmss)}

• Add disk to VM via Enterprise Service Catalog
• Rename new volume group to esosvg, or add new disk to existing esosvg volume group
• LV esosvg/home mounted at /home
• LV esosvg/mariadb mounted at /var/lib/mysql
• LV esosvg/opt mounted at /opt (only if rootvg /opt isn't sufficient)
• LV esosvg/arc mounted at /opt/arc
• Alternative: LV esosvg/home mounted at /home with all other locations sym-linked under /home

• Update system with yum update
• Lay out storage per conventions
• Install/enable EPEL: dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
• Install newterm
• Create users
• Change /etc/sudoers to switch to NOPASSWD: wheel group entry
• Install basic packages
  • yum install wget htop vim gvim git
  • yum groupinstall “X Window System”
  • yum install dejavu-sans-mono-fonts.noarch
  • yum install xorg-x11-fonts-misc
  • Optional: yum groupinstall “Fonts” for all fonts
  • yum install tightvnc-server
  • yum install tightvnc
  • yum install xorg-x11-server-devel
• Install …
  • yum install java-1.8.0-openjdk-devel
  • yum install mariadb-server
  • yum install mariadb-devel
  • yum install python-devel
  • yum install MySQL-python
  • yum install httpd
  • yum install php
  • yum install php-mysql
  • yum install php-ldap
  • yum install php-gd
  • yum install gd-devel
• Install Maven
• Install InfluxDB (requires EPEL)
• Install Grafana
• Install Jenkins

• Ensure SSH key and known_hosts are populated for self-connection
• Add plugins
• Add credentials, use “Git” provider (not “GitHub”)
• Configure global tools section (maven, etc…)
• Add environment variables in Global properties
  • M3_HOME = /usr/maven
  • PATH = /var/lib/jenkins/bin:/usr/maven/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin
  • Optional: JAVA_HOME = /usr/java
• Addition packages:
  • yum install python-pip
  • pip install –upgrade pip
  • pip install jenkinsapi
  • Correct permissions, if necessary, at /usr/lib/python2.7/site-packages for:
    • jenkinsapi
    • six
    • pytz

Previously-mentioned package install via yum installs the software
Configuration is in /etc/my.cnf and /etc/my.cnf.d:

• Setting the datadir attribute to where database storage is to live is advisable, to separate it from /var/lib/mysql, which is where the socket is created
• Trying to symbolic-link /var/lib/mysql to some other location causes permissions quirks
• For some reason, /var/run/mariadb is sometimes not created - to remediate:

  mkdir /var/run/mariadb
  chown mysql:mysql /var/run/mariadb
  chmod 775 /var/run/mariadb
  chmod g+s /var/run/mariadb
  cd /var/run/mariadb ; chcon -t var_run_t .

• Make sure /var/lib/mysql has mysqld_db_t type, in the directory: chcon -t mysqld_db_t .

Previously-mentioned package install via yum installs the software Edit /etc/httpd/conf/httpd.conf:

• Set DocumentRoot to suitable location, ensure location exists and resides in suitable storage

Open firewall for Apache:

firewall-cmd --permanent --zone=public --add-port=80/tcp
firewall-cmd --reload

Perform the typical systemctl enable httpd and service httpd start for startup

Resource Clustering InfluxDB ...

Create the file /etc/yum.repos.d/influxdb.repo with this content:

[influxdb]
name = InfluxDB Repository - RHEL $releasever
baseurl = https://repos.influxdata.com/rhel/$releasever/$basearch/stable
enabled = 1
gpgcheck = 1
gpgkey = https://repos.influxdata.com/influxdb.key

… then, do a yum update and yum install influxdb
Open firewall for InfluxDB:

firewall-cmd --permanent --zone=public --add-port=8086/tcp
firewall-cmd --permanent --zone=public --add-port=8083/tcp
firewall-cmd --reload

Edit /etc/influxdb/influxdb.conf and:

• In the [http] section:
  • Set enabled = true
  • Change the bind = :8086 entry, if needed
  • Set unix-socket-enabled = true
  • Change bind-socket = value to /var/run/influxdb/influxdb.sock
• In the [udp] section:
  • Set enabled = true
  • Change the bind = :8089 entry, if needed

Ensure /var/run/influxdb and /var/lib/influxdb exist and are owned correctly

• For some reason, /var/run/influxdb is sometimes not created - to remediate:

  mkdir /var/run/influxdb
  chown influxdb:influxdb /var/run/influxdb
  chmod 775 /var/run/influxdb
  chmod g+s /var/run/influxdb
  cd /var/run/influxdb ; chcon -t var_run_t .

Remember to systemctl enable influxdb and service influxdb start

See: InfluxDB auth

create user admin with password 'somepassword' with all privileges;
create user jmeter with password 'somepassword';
create database jmeter;
grant [READ,WRITE,ALL] on "jmeter" TO "jmeter";
revoke [READ,WRITE,ALL] on "jmeter" from "jmeter";

Create the file /etc/yum.repos.d/grafana.repo with this content:

[grafana]
name=grafana
baseurl=https://packagecloud.io/grafana/stable/el/7/$basearch
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packagecloud.io/gpg.key https://grafanarel.s3.amazonaws.com/RPM-GPG-KEY-grafana
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt

… then, do a yum update and yum install grafana
Open firewall for Grafana:

firewall-cmd --permanent --zone=public --add-port=3000/tcp
firewall-cmd --reload

Remember to systemctl enable grafana-server and service grafana-server start
Log in (default login is admin with password admin) and change password, add user(s)
Install the InfluxDB datasource plugin (find in plugins section, follow instructions)

Edit /etc/grafana/grafana.ini:

[auth.ldap]
enabled = true
config_file = /etc/grafana/ldap.toml
allow_sign_up = true

Edit /etc/grafana/ldap.toml:

• Under servers.attributes section, make sure username is set to sAMAccountName for Active Directory
• Under servers.group_mappings section, group_dn should be cn=bluejay_adm,cn=Users,dc=ms,dc=ds,dc=uhc,dc=com
• Editor role: cn=github_users,cn=Users,dc=ms,dc=ds,dc=uhc,dc=com
• Host: “ADC21101.ms.ds.uhc.com”
• ssl_skip_verify = true
• bind_dn: “oaptbot@ms.ds.uhc.com”
• bind_password: 'BlueJay1!'
• search_filter: “(sAMAccountName=%s)”
• search_base_dns: [“cn=Users,dc=ms,dc=ds,dc=uhc,dc=com”]

See Jenkins Topics

• Ensure central repository at /var/www/html/pub/eptrepo/jmeter is populated, and create SSL cert
  • Edit the create-rmi-keystore.sh script, changing the validity to 99999 and run to gen cert
  • Ensure cert file rmi_keystore.jks has same ownership/permissions/seattr as other files
• On Jenkins slave, perform basic installation, update, and pkgs for: wget, htop, vim, git, and OpenJDK
• Populate /etc/hosts
• Create bluejay login

  groupadd -g 39092 bluejay
  useradd -u 441849 -g 39092 -d /home/bluejay -s /bin/bash -c "Bluejay" -m bluejay
  passwd bluejay
  chage -I -1 -m 0 -M 99999 -E -1 bluejay

• Extended storage setup:

  lvcreate -l 100%FREE -n bluejay rootvg
  mkfs -t ext4 /dev/rootvg/bluejay
  blkid /dev/rootvg/bluejay >> /etc/fstab
  vi /etc/fstab
  ls -lZd /home/bluejay/.
  mkdir -p ~/tmp
  mv /home/bluejay/.??* ~/tmp/
  chmod 755 /home/bluejay
  mount /home/bluejay
  chown bluejay:bluejay /home/bluejay/.
  chmod 775 /home/bluejay/.
  chcon -u unconfined_u -t user_home_dir_t /home/bluejay/.
  mv ~/tmp/.??* /home/bluejay/

• Populate bluejay login's ~/.ssh directory:

  cd ~bluejay ; mkdir .ssh ; cd .ssh
  ssh-keygen -t rsa -f id_rsa -C bluejay@hostname
  cp -a id_rsa.pub authorized_keys
  chown -R bluejay:bluejay .
  chcon -R -u unconfined_u -t ssh_home_t .
  chmod 700 . ; chmod 600 *

• Copy public key to central staging location:

  scp ./id_rsa.pub someone@somewhere:/home/someone/dot-ssh/bluejay_host.pub

• Create jenkins login

  groupadd -g 822 jenkins
  useradd -u 990 -g 822 -d /var/lib/jenkins -s /bin/bash -c "Jenkins" jenkins
  usermod -a -G bluejay jenkins
  usermod -a -G jenkins bluejay
  chcon -u system_u -t var_lib_t /var/lib/jenkins/.
  chmod 750 /var/lib/jenkins/.

• Populate jenkins login's ~/.ssh directory:

  cd ~jenkins ; mkdir .ssh ; cd .ssh
  ssh-keygen -t rsa -f id_rsa -C jenkins@hostname
  cp -a id_rsa.pub authorized_keys
  chown -R jenkins:jenkins .
  chcon -R -u unconfined_u -t ssh_home_t .
  chmod 700 . ; chmod 600 *

• Copy public key to central staging location, and make sure it gets populated into Bluejay web server's bluejay login ~/.ssh/authorized_keys file:

  scp ./id_rsa.pub someone@somewhere:/home/someone/dot-ssh/jenkins_host.pub

• Create Jenkins agent workspace:

  cd ~bluejay ; mkdir -p jenkins
  chown bluejay:jenkins ./jenkins/.
  chmod 770 ./jenkins/.
  chmod g+s ./jenkins/.
  chcon -u system_u -t var_lib_t ./jenkins/.

• Populate Jenkins controller's jenkins login SSH key into bluejay login's authorized_keys
• Populate jenkins known_hosts file on Jenkins host

• msamud1: uid=7301
• jenkins: uid=990, gid=822
• bluejay: uid=441849, gid=39092
• mosquito: uid=263407, gid=28606
• canary: uid=783478, gid=31708
• apache: uid=48, gid=1739
• mantis: uid=17236, gid=?
• wildfly: uid=185, gid=185 wildfly: uid=282, gid=28202
• thorium: gid=34625
• perfmgmt: gid=4035

## RHEL/CentOS 7 64-Bit ##
# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# rpm -ivh epel-release-latest-7.noarch.rpm

yum –enablerepo=epel info pkgname
yum –enablerepo=epel install pkgname

If /opt happens to be an XFS filesystem, resizing can be done “live”. Otherwise, in order to unmount /opt to manipulate it, all processes holding files open on the filesystem have to be shut down, including:

/etc/init.d/opsware-agent stop
/etc/init.d/vasd stop
service postfix stop
service dbus stop

Edit /etc/nsswitch.conf to comment out lines with vas4 in them, then kill -HUP <pid-of-nss-process>

Perform the whole thing in reverse when /opt is resized

This problem: http://openssl.6102.n7.nabble.com/DEK-info-FIPS-openssl-td23289.html
… is resolved by downloading cert from Venafi in PKCS#8 format