%
% Ansible.tex
%
% Fork Sand IT Manual
%
% Copyright (C) 2018, Fork Sand, Inc.
% Copyright (C) 2017, Jeff Moe
%
% This document is licensed under the Creative Commons Attribution 4.0
% International Public License (CC BY-SA 4.0) by Fork Sand, Inc.
%
\section{Ansible Cloud Management}
Use \texttt{ansible} for management of servers.

\begin{itemize}
 \item Ansible ---
  Website: \\ \url{https://ansible.com}
 \item Ansible Github ---
  Repo: \\ \url{https://github.com/ansible/ansible.git}
 \item DebOps: \url{https://docs.debops.org/en/latest/index.html}
\end{itemize}

\subsection{Glossary}
An \texttt{Ansible playbook} is an organized unit of scripts that defines
work for a server configuration managed by the automation tool \texttt{ansible}.

\subsection{Build Ansible Debian Package}\label{ssec:bansdpac}
The version of Ansible is 2.2 in Debian 9 (stable/Stretch).
The current stable release is Ansible 2.4.
Below documents how to build a 2.4 package for Debian 9.

\begin{minted}{sh}
# To build a Debian package:
# Check here for latest version:
# https://packages.debian.org/sid/ansible
wget http://http.debian.net/debian/pool/main/a/ansible/ansible_2.4.0.0+dfsg-1.debian.tar.xz
wget http://http.debian.net/debian/pool/main/a/ansible/ansible_2.4.0.0+dfsg.orig.tar.gz

# Install some deps
apt-get install debhelper python-all python-crypto python-setuptools python-yaml asciidoc python-nose python-passlib dh-python 

tar xf ansible_2.4.0.0+dfsg.orig.tar.gz
cd ansible-2.4.0.0/
tar xf ../ansible_2.4.0.0+dfsg-1.debian.tar.xz

# Update version:
echo -n " -- Jeff Moe <moe@blagblagblag.org>  " ; date "+%a, %d %b %Y %H:%M:%S %z"
vim debian/changelog

dpkg-buildpackage -rfakeroot -S -uc -us -sa
dpkg-buildpackage -rfakeroot -b -uc

# That will produce this file to be installed:
dpkg -i ansible_2.4.0.0+dfsg-2_all.deb
apt-get -f install

# If you want the Ansible git archive:
git clone https://github.com/ansible/ansible.git --recursive
\end{minted}

\subsection{Ansible Initial Configuration}
Here is how to set up Ansible after initially installing it.
This is run on the system adminstrator's workstation.

\begin{minted}{sh}
# Quick and dirty test by setting up a host and running `uptime`.
mkdir -p ~/.ansible
echo ns1 > ~/.ansible/hosts
ansible -i ~/.ansible/hosts ns1 -a 'uptime'
\end{minted}

~/.ansible.cfg
\begin{minted}{sh}
[defaults]
inventory = $HOME/.ansible/hosts

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=300s
pipelining = True
\end{minted}

To generate a full list of \texttt{ns} hosts, run the script:

\begin{minted}{sh}
cd source/resources/servers
echo "[ns]" > ~/.ansible/hosts
./ns-serverlist-ansible.sh >> ~/.ansible/hosts
ansible -i ~/.ansible/hosts ns -a "uptime"
# Find the failed hosts and remove them from ~/.ansible/hosts.
echo "[ns]" > ~/.ansible/hosts.tmp
ansible -f 32 -i ~/.ansible/hosts ns -a "echo" | grep ^ns | grep SUCCESS | cut -f 1 -d " " | sort -V >> ~/.ansible/hosts.tmp
mv ~/.ansible/hosts ~/.ansible/hosts.old
mv ~/.ansible/hosts.tmp ~/.ansible/hosts
# Test it works:
ansible -i ~/.ansible/hosts ns -a "uptime"
# To get a ton of info about each host:
ansible -i ~/.ansible/hosts ns -m setup
\end{minted}

XXX Fix, make sure everyone has \texttt{/usr/bin/python} available for \texttt{ansible}:

\begin{minted}{sh}
# XXXX SOME HOSTS DON'T HAVE /usr/bin/python
# JUST PYTHON3. HOSTS THAT DIDN'T HAVE /usr/bin/python:
# (ALL OVH)
# ns14 ns15 ns21 ns22
apt install python python-minimal
\end{minted}

Set up some playbooks, grab examples:

\begin{minted}{sh}
git clone https://github.com/ansible/ansible-examples.git
cd ansible-examples/
\end{minted}

\section{Ansible Debian Mail}
Assuming \texttt{ansible} is built according to section \ref{ssec:bansdpac}
on p.\pageref{ssec:bansdpac}, the following requirements are met.

\subsection{Requirements}
The following applications are required to utilize this this section objectives.
Ansible can be installed using Python PIP. 
\begin{itemize}
    \item \texttt{Ansible} 2.4.x+
    \item \texttt{Python} 2.7.x+   
        \textcolor[rgb]{0.80,0.00,0.00}{Todo clarify confusion over version requirements}
\end{itemize}

\subsection{Quick Start}
The following steps will help quickly set up and execute this section objectives.

\texttt{Project Configuration}

The following files need to be edited and configured before executing this playbook.

\begin{table}[!htb]
    \caption{Files to be edited} % \label{tab:tech}
    \begin{tabular}{|l|l|}
        \hline
        \multicolumn {1}{|l|}{ File}&
        \multicolumn {1}{l|}{ Description} \\ \hline
         groups\char`_vars/all.yml & Server credential information and domain variables \\ \hline
         inventory.yml & List of server IPs to connect to \\ \hline
    \end{tabular}
\end{table}

\texttt{Playbook Execution}

After having configured the server credentials and added the server IP to the inventory, use the following command to execute the playbook.

\begin{minted}{sh}
ansible-playbook -i inventory.yml site.yml -vvv
\end{minted}

\subsection{Project Structure}
The following tree depicts the high level structure of this Ansible project.

\begin{minted}{sh}
|-- ansible.cfg
|-- group_vars
|    -- all.yml
|-- inventory.yml
|-- LICENSE.AGPLv3
|-- LICENSE.GPLv3
|-- README.md
|-- roles
|   |-- dkim_configuration
|   |-- dovecot_configuration
|   |-- fail2ban_configuration
|   |-- letsencrypt_configuration
|   |-- mikegleasonjr.firewall
|   |-- outputs
|   |-- postfix_configuration
|   |-- server_tasks
|   |-- spamassassin_configuration
|    -- sqlgrey_configuration
|-- playbook_execution.log
 -- site.yml
\end{minted}

\texttt{File and Directory Descriptions}

The following table consists of a description of what each file and directory stands for.

\begin{table}[!htb]
    \caption{File and Directory Descriptions} % \label{tab:tech}
    \begin{tabular}{|l|l|}
        \hline
        \multicolumn {1}{|l|}{ Name}&
        \multicolumn {1}{l|}{ Description} \\ \hline
         site.yml & Master playbook. Executes all roles in sequential order \\ \hline
         inventory.yml & Inventory file containing server IP addresses \\ \hline
         ansible.cfg & Ansible configuration file for various Ansible options. \\ \hline
         group\char`_vars/ & Group\char`_vars directory contains variable files for the entire group. \\
          & The files are named according to the group name. 'all.yml' = group 'all' \\ \hline
         group\char`_vars/all.yml & Group variables for the 'all' group. Contains server connection \\
          & information along with domain variables \\ \hline
         roles/ & Directory containing all roles needed by this project \\ \hline
    \end{tabular}
\end{table}

\texttt{Role descriptions}

The following table consists of descriptions of each role and their purpose. The roles listed below are listed in the required order of execution to ensure successful completion of the playbook.

\begin{table}[!htb]
    \caption{Role descriptions} % \label{tab:tech}
    \begin{tabular}{|l|l|l|}
        \hline
        \multicolumn {1}{|l|}{Role Name}&
        \multicolumn {1}{|l|}{Role Description}&
        \multicolumn {1}{l|}{Depends on} \\ \hline
         server\char`_tasks & This roles performs all server tasks. Updating & mikebleasonjr.firewall \\
                      & server, configuring SSH, disable IPv6, etc. & \\
                      & Depends on the mikegleasonjr.firewall role. & \\ \hline
         mikegleasonjr.firewall & This role set up iptables rules. It is called & None \\
                                & and ran by the server\char`_tasks roles. & \\ \hline
         letsencrypt\char`_configuration & This role installs and executes let's encrypt & None \\ \hline
         postfix\char`_configuration & This roles installs postfix, configures postfix & letsencrypt\char`_configuration \\
                                     & using postconf, and sets up virtual file, & \\
                                     & master.cf file, and aliases file & \\ \hline
         dkim\char`_configuration & This roles installs OpenDKIM, OpenDMARC & None \\ 
                                  & and configures them. & \\ \hline
         dovecot\char`_configuration & This role installs and configures dovecot & letsencrypt\char`_configuration \\ \hline
         spamassassin\char`_configuration & This role installs spamassassin. & None \\ \hline
         sqlgrey\char`_configuration & This role installs sqlgrey. & None \\ \hline
         fail2ban\char`_configuration & This role installs fail2ban. & None \\ \hline
         outputs & This role gathers DNS information for the & None \\
                 & SPF, DMARC, and DKIM records and & \\
                 & outputs them to the screen. & \\ \hline
    \end{tabular}
\end{table}

\subsection{Ansible Logging}
Execution processes described in this section are automatically logged to a file called `playbook-execution.log` in the root directory of the project. The path to this log file can be changed by editing `ansible.cfg` in the project root directory and specifying a different path.

\subsection{Troubleshooting}
Ansible has a built in debug output. Simply run Ansible with a `-v`. There are 5 levels of debug output and they are denoted by the number of v's listed. Each level up provide more debug output than the level before it.

\begin{minted}{sh}
Level 1: `-v`

Level 2: `-vv`

Level 3: `-vvv`

Level 4: `-vvvv`

Level 5: `-vvvvv`
\end{minted}

Example execution with level 3 debug output:

\begin{minted}{sh}
ansible-playbook -i inventory.yml site.yml -vvv
\end{minted}

\section{Ansible Gitea}
Assuming \texttt{ansible} is built according to section \ref{ssec:bansdpac}
on p.\pageref{ssec:bansdpac}, the following requirements are met.

\subsection{Requirements}
The following applications are required to utilize this this section objectives.
Ansible can be installed using Python PIP. 
\begin{itemize}
    \item \texttt{Ansible} 2.4.x+
    \item \texttt{Python} 2.7.x
\end{itemize}

\subsection{Quick Start}
The following steps will help quickly set up and execute this section objectives.

\texttt{Project Configuration}

The following files need to be edited and configured before executing this playbook.

\begin{table}[!htb]
    \caption{Files to be edited} % \label{tab:tech}
    \begin{tabular}{|l|l|}
        \hline
        \multicolumn {1}{|l|}{ File}&
        \multicolumn {1}{l|}{ Description} \\ \hline
         roles/gitea/default/main.yml & Variables for Gitea configuration \\
          & (default - standalone with sqlite) \\ \hline
         roles/nginx/default/main.yml & Variables for Nginx and Letsencrypt configuration \\ \hline
         inventory.yml & List of server IPs to connect to \\ \hline
    \end{tabular}
\end{table}

\qquad \\
\texttt{Playbook Execution}

After having configured the server credentials and added the server IP to the inventory, use the following command to execute the playbook.

\begin{minted}{sh}
ansible-playbook -i inventory.yml site.yml -vvv
\end{minted}

\subsection{Project Structure}
The following tree depicts the high level structure of this Ansible project.

\begin{minted}{sh}
|-- inventory.yml
|-- LICENSE.AGPLv3
|-- LICENSE.GPLv3
|-- README.md
|-- roles
|   |-- gitea
|    -- nginx
|-- playbook_execution.log
 -- site.yml
\end{minted}

\texttt{File and Directory Descriptions}

The following table consists of a description of what each file and directory stands for.

\begin{table}[!htb]
    \caption{File and Directory Descriptions} % \label{tab:tech}
    \begin{tabular}{|l|l|}
        \hline
        \multicolumn {1}{|l|}{ Name}&
        \multicolumn {1}{l|}{ Description} \\ \hline
         site.yml & Master playbook. Executes all roles in sequential order \\ \hline
         inventory.yml & Inventory file containing server IP addresses \\ \hline
         ansible.cfg & Ansible configuration file for various Ansible options. \\ \hline
         roles/ & Directory containing all roles needed by this project \\ \hline
    \end{tabular}
\end{table}

\qquad \\

\texttt{Role descriptions}

The following table consists of descriptions of each role and their purpose. The roles listed below are listed in the required order of execution to ensure successful completion of the playbook.

\begin{table}[!htb]
    \caption{Role descriptions} % \label{tab:tech}
    \begin{tabular}{|l|l|l|}
        \hline
        \multicolumn {1}{|l|}{Role Name}&
        \multicolumn {1}{|l|}{Role Description} \\ \hline
         gitea & This roles performs installation and configuration of Gitea server \\ \hline
         nginx & This roles performs installation and configuration of Nginx server \\ \hline
    \end{tabular}
\end{table}

\texttt{Role parameters}

\qquad \\

\texttt{\qquad Gitea role parameters}
\begin{minted}{sh}
 # Application name
 gitea_app_name: "Gitea"
 # Application gitea_user_repo_limit
 gitea_user: "gitea"
 # Application home
 gitea_home: "/var/lib/gitea"
 # Repo Limit
 gitea_user_repo_limit: -1
 # Domain Name (FOR REVER PROXY LEAVE AS DEFAULT)
 gitea_http_domain: localhost
 # Gitea url (FOR REVER PROXY LEAVE AS DEFAULT)
 gitea_root_url: http://localhost:3000
 # Protocol (FOR REVER PROXY LEAVE AS DEFAULT)
 gitea_protocol: http
 # listen IP (FOR REVER PROXY LEAVE AS DEFAULT)
 gitea_http_listen: 127.0.0.1
 # Listen port (FOR REVER PROXY LEAVE AS DEFAULT)
 gitea_http_port: 3000
 # HTTP git Options
 gitea_disable_http_git: false
 # Offline mode options
 gitea_offline_mode: true
\end{minted}

\qquad \qquad DB details

\begin{minted}{sh}
 # DB Type 'mysql', 'postgres' or 'sqlite3'
 gitea_db_type: sqlite3
 # DB host
 gitea_db_host: 127.0.0.0:3306
 # DB name
 gitea_db_name: root
 # DB username
 gitea_db_user: gitea
 # DB password
 gitea_db_passord: lel
 # DB ssl options
 gitea_db_ssl: disable
 # DB path (Not needed for postgres and mysql hash it in template file )
 gitea_db_path: "{{ gitea_home }}/data/gitea.db"
\end{minted}

\qquad \qquad SSH Details

\begin{minted}{sh}
 # SSH Listen IP
 gitea_ssh_listen: 0.0.0.0
 # SSH domain
 gitea_ssh_domain: localhost
 # SSH options
 gitea_start_ssh: true
 # SSH post
 gitea_ssh_port: 2222

\qquad \qquad Gitea secret key

\begin{minted}{sh}
 # gitea key (GENERATE A NEW KEY)
 gitea_secret_key: T0pS3cr31
\end{minted}

\qquad \qquad General Settings

\begin{minted}{sh}
 # User email settings
 gitea_show_user_email: false
 # User avatar settings
 gitea_disable_gravatar: true
 # User register options
 gitea_disable_registration: false
 # User signup options
 gitea_require_signin: true
 # User captcha options
 gitea_enable_captcha: true
\end{minted}

\qquad \\

\texttt{\qquad Nginx roles parameters}
\begin{minted}{sh}
 # Domain name for the server
 nginx_domain_name: "test.hostnats.com"
 # Gitea listening port
 gitea_http_port: 3000
 # letsencrypt email address
 letsencrypt_email: "test@example.com"
\end{minted}

\subsection{Ansible Logging}
Execution processes described in this section are automatically logged to a file called `playbook-execution.log` in the root directory of the project. The path to this log file can be changed by editing `ansible.cfg` in the project root directory and specifying a different path.

\subsection{Troubleshooting}
Ansible has a built in debug output. Simply run Ansible with a `-v`. There are 5 levels of debug output and they are denoted by the number of v's listed. Each level up provide more debug output than the level before it.

\begin{minted}{sh}
Level 1: `-v`

Level 2: `-vv`

Level 3: `-vvv`

Level 4: `-vvvv`

Level 5: `-vvvvv`
\end{minted}

Example execution with level 3 debug output:

\begin{minted}{sh}
ansible-playbook -i inventory.yml site.yml -vvv
\end{minted}