%
% Proxmox.tex
%
% Fork Sand IT Manual
%
% Copyright (C) 2018, Fork Sand, Inc.
% Copyright (C) 2017, Jeff Moe
% Copyright (C) 2017 Aleph Objects, Inc.
%
% This document is licensed under the Creative Commons Attribution 4.0
% International Public License (CC BY-SA 4.0) by Fork Sand, Inc.
%
% XXX TODO: Proxmox-GUI-login.png 

\section{Overview}
Proxmox is a virtual machine manager. 

The private cloud deployment will be based on Proxmox version 5.x.

Documentation:
\url{https://pve.proxmox.com/wiki/Documentation}

\begin{figure}[!htb]
\includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]{www-proxmox.png}
 \caption{Proxmox Website}
 \label{fig:www-proxmox}
\end{figure}

\begin{itemize}
 \item Website: \\ \url{https://proxmox.com/}
 \item Debian Stretch ISO (XXX check version): \\
   \url{http://download.proxmox.com/iso/proxmox-ve_5.1-3.iso}

 \item Complete operating system (Debian Linux, 64-bit)
 \item The Proxmox VE installer, which partitions the hard
drive(s) with ext4, ext3, xfs or ZFS and installs the
operating system.
 \item Proxmox VE kernel (Linux) with LXC and KVM support
Proxmox VE Administration Guide 9/309
 \item Complete toolset for administering virtual machines,
containers and all necessary resources
 \item Web based management interface for using the toolset
 \item Debian Stretch admin guide: \\
   \url{https://pve.proxmox.com/pve-docs/pve-admin-guide.pdf}
\end{itemize}


The following servers will be deployed to host Proxmox and the KVMs:

\begin{itemize}
 \item \texttt{sf-001} --- Virtual Machine Node 1
 \item \texttt{sf-002} --- Virtual Machine Node 2
 \item \texttt{sf-003} --- Virtual Machine Node 3
 \item \texttt{sf-004} --- Virtual Machine Node 4
\end{itemize}

%\subsection{Virtual Machine Servers}
%KVM virtual machine servers. Fast CPU, with lots of RAM. Uses Ceph to store
%virtual images.
%
%\subsection{Proxmox Web GUI Servers}
%A Proxmox's Web GUI for administration of the \gls{cluster}.

\subsection{Virtual Machine Nodes}
Virtual machine nodes. Fast CPU, with lots of RAM. Uses Ceph to store
virtual images.

Every node includes a Proxmox's Web GUI service for administration of the \gls{cluster}.
Any nodes included into the \gls{cluster} may be configured by requesting to any node's GUI.

\begin{figure}[!htb]
\includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]{proxmox-gui.png}
 \caption{Proxmox Sunstone Web Admin GUI}
 \label{fig:proxmox-gui}
\end{figure}


\begin{minted}{sh}
echo "deb http://download.proxmox.com/debian/pve stretch pve-no-subscription" \
> /etc/apt/sources.list.d/pve-install-repo.list
wget http://download.proxmox.com/debian/proxmox-ve-release-5.x.gpg \
-O /etc/apt/trusted.gpg.d/proxmox-ve-release-5.x.gpg
apt-get update
apt-get -y dist-upgrade --download-only
DEBIAN_FRONTEND=noninteractive apt-get -y \
-o Dpkg::Options::="--force-confdef" \
-o Dpkg::Options::="--force-confnew" dist-upgrade
apt-get -y install ksm-control-daemon proxmox-veupdate-grub
apt remove os-prober
\end{minted}

\section{Bugs}
Things that are bugs or at least aren't configured correctly.

\section{GUI Configuration}
At this point, you should have the Proxmox server up and running.

\textcolor[rgb]{0.80,0.00,0.00}{
Todo check related, modify/replace unrelated
}

%See scripts in \texttt{source/resources/ns24} for automation.
\begin{enumerate}
\item Set up Linux Bridge (see fig. \ref{fig:sf-002linuxbridge} p.\pageref{fig:sf-002linuxbridge}):
 %Code: \\ \texttt{ssh -N -C -L 9869:localhost:9869 ns24}
\item In workstation, open browser to urls: \\
URL: \url{http://localhost:8001/}, for sf-001 \\
URL: \url{http://localhost:8002/}, for sf-002 \\
URL: \url{http://localhost:8003/}, for sf-003 \\
URL: \url{http://localhost:8004/}, for sf-004 \\
See example at fig. \ref{fig:proxmox-gui-port}:
\begin{figure}[!htb]
    \includegraphics[keepaspectratio=true,height=1.10\textheight,width=1.00\textwidth,angle=0]{shark2/23.png}
    \label{fig:proxmox-gui-port}
    \caption{Browse sf-002 node, visible port No.}
\end{figure}
Info: This goes through https with self-signed certificate.
\item \texttt{Hostname} Changing the hostname and IP is not possible after
    \gls{cluster} creation. Unlike OpenNebula.
%\item Click \texttt{Infrastructure}.
%\item Click \texttt{Hosts}.
%\item Click The \texttt{+} plus icon.
%\item Enter the hostname of the KVM server you want to use, such as the Sunstone server itself.
% \texttt{Type: KVM}
% \texttt{Hostname: ns24}
%\item Click \texttt{Create}.
%\item Repeatedly hit the reload button that's the two arrows in a circle, as it goes thru
% stages of setup, starting at \texttt{INIT}.
\item Confirm status is \texttt{ON}.
\end{enumerate}

\section{GUI Deploy Image}
\textcolor[rgb]{0.80,0.00,0.00}{Todo check related, modify/replace unrelated}
\begin{minted}{sh}
This is a quick and dirty way to deploy a first test image.
NOTE: It is note privacy aware, as it pulls the image from the
Proxmox ``store''.
\end{minted}

\textcolor[rgb]{0.80,0.00,0.00}{Todo check related, modify/replace unrelated}
\begin{minted}{sh}
How to deploy an image from the Proxmox App store:
\begin{enumerate}
 \item Click \texttt{Storage}.
 \item Click \texttt{Apps}
 \item Click \texttt{Debian 9 - KVM}.
 \item Click on the icon that is a cloud with an arrow in it. This downloads it to Proxmox.
 \item Select a datastore by clicking the \texttt{default} line.
 \item Leave name and all that the same, and click \texttt{Download}.
 \item Click \texttt{Images} under \texttt{Storage} in the left column.
 \item Hit the refresh icon repeatedly.
 \item When \texttt{Status} is \texttt{READY}, it is good to go.
 \item Click \texttt{Templates} in the left column.
 \item Click \texttt{VMs}.
 \item Click \texttt{Debian 9 - KVM}.
 \item Click \texttt{Instantiate}.
 \item \texttt{VM Name} enter \texttt{deb9}.
 \item \texttt{Number of instances} enter \texttt{1}.
 \item \texttt{Memory} enter \texttt{768}.
 \item \texttt{CPU} enter \texttt{1}.
 \item Click the slider to \texttt{Instantiate as persistent}.
 \item Click \texttt{Instantiate}.
 \item Click \texttt{Instances} in the left column.
 \item Click \texttt{VMs}.
 \item Click the reload icon, repeat.
 \item It is good when \texttt{Status} is \texttt{RUNNING}.
 \item Set up an \texttt{ssh} tunnel so VNC can be used:
  \texttt{ssh -N -C -L 29876:localhost:29876 ns24}
 \item Click on the little monitor icon to launch VNC.
 \item Look at booted up screen at \texttt{login:} prompt.
 \item This means a Debian KVM booted up and the VNC is working.
  There is no password for the \texttt{root} account, only \texttt{ssh} is available.
  So without network setup, you can't really do anything with this image.
  Booted, it just shows it works.
\end{enumerate}

I think delete this section, it would go before the \texttt{Templates} above.:
\begin{enumerate}
 \item Click \texttt{Debian 9 - KVM}.
 \item PROBABLY NO: Click \texttt{Clone} to make a local copy.
 \item PROBABLY NO: It will say \texttt{Copy of Debian 9 - KVM}, leave as-is, click \texttt{Clone}.
 \item Click on the icon with three dots.
 \item Click \texttt{Make Persistent}.
 \item Click on the icon with three dots.
 \item Click \texttt{Enable}.
\end{enumerate}
\end{minted}

\section{Proxmox Networking}
Create --> Linux Bridge: vmbr0

XXX best way for this server? No subnet.

\textcolor[rgb]{0.80,0.00,0.00}{taken from sf-004-bootstrap}

\begin{minted}{sh}
source /etc/network/interfaces.d/*
auto enp1s0f1
iface enp1s0f1 inet static
        address 70.39.103.210/29
        gateway 70.39.103.209
        dns-nameservers 208.67.222.222
        dns-search forksand.com
\end{minted}

As user \texttt{jebba}, on the server, run this to generate a key.
Then paste that key into Sunstone under "SSH Public Key".

\begin{minted}{sh}
ssh-keygen -t ed25519
\end{minted}

\begin{minted}{sh}
# XXX test. Use this IP and interface, so no 192.168.0.0 but real IPs.
# Comment this out:
auto eth0:27
iface eth0:27 inet static
address 174.128.229.158
netmask 255.255.255.224
gateway 174.128.229.129
\end{minted}

XXX Check if IP forwarding is needed in \texttt{/etc/sysctl.conf}.

If things are set up to use a bridge and 192.168.100.100,
\texttt{iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE}
Will bring things up to NAT.


XXX The port forwarding is forwarding all port 53 to guest at the moment.