FreeBSD on Google Compute Engine
Steps Toward The Goal
On the previous page
I showed how to set up a
Google Compute Engine
virtual machine within the
Google Cloud Platform.
for good performance, stability, and minimal complexity.
In this stage I will set up the FreeBSD system:
1: Add an unprivileged user.
2: Set up SSH authentication with keys only.
3: Check the file systems and networking.
4: Install some packages: Apache, PHP, and others.
5: Fix a clock problem.
Try Google Cloud Platform and receive $50 On following pages I will install free Let's Encrypt TLS certificates for both RSA and ECC, and then adjust the Apache configuration for a good score from the authoritative Qualys server analysis.
Adding a User
You can connect in over SSH as a user using cryptographic
That user can become
root in an
interactive session with:
Now you can create a user with a name of your choice,
Do not assign passwords to any users!
Continue using SSH keys for all authentication into your system. Automated password-guessing attacks constantly arrive from all across the Internet. There is no need for the risk posed by supporting weak password authentication. You can find the list of supported authentication keys by running this command on your server:
$ ssh -Q key ssh-ed25519 firstname.lastname@example.org ssh-rsa ssh-dss ecdsa-sha2-nistp256 ecdsa-sha2-nistp384 ecdsa-sha2-nistp521 email@example.com firstname.lastname@example.org email@example.com firstname.lastname@example.org email@example.com
If you add your new user to group
they will be able to become
by simply running the command
because of the contents of
File Systems, Device Detection, and Networking
Let's look at the file systems, and then see what's in the kernel ring buffer. The following is after I had installed the web site, which takes up 1.7 GB.
dmesg output is edited down to show the
CPU, memory, timer devices, and the disk.
$ df -hT Filesystem Type Size Used Avail Capacity Mounted on /dev/gpt/rootfs ufs 28G 5.5G 20G 21% / devfs devfs 1.0K 1.0K 0B 100% /dev $ swapctl -l Device: 1024-blocks Used: /dev/gpt/swapfs 1048576 15788 $ dmesg | less ---<<BOOT>>--- Copyright (c) 1992-2021 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261f: Fri Apr 9 04:24:09 UTC 2021 firstname.lastname@example.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 FreeBSD clang version 11.0.1 (email@example.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c3fe) VT(vga): text 80x25 CPU: Intel(R) Xeon(R) CPU @ 2.20GHz (2200.06-MHz K8-class CPU) Origin="GenuineIntel" Id=0x406f0 Family=0x6 Model=0x4f Stepping=0 Features=0x1f83fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,SS,HTT> Features2=0xfefa3203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=0x121<LAHF,ABM,Prefetch> Structured Extended Features=0x1c2ffb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,FDPEXC,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG,RDSEED,ADX,SMAP> XSAVE Features=0x1<XSAVEOPT> TSC: P-state invariant Hypervisor: Origin = "KVMKVMKVM" real memory = 643825664 (614 MB) avail memory = 586088448 (558 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: <Google GOOGAPIC> random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 <Version 1.1> irqs 0-23 Timecounter "TSC-low" frequency 1100027555 Hz quality 1000 KTLS: Initialized 1 threads random: entropy device external interface [... output deleted ...] vtvga0: <VT VGA driver> cryptosoft0: <software crypto> aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS> acpi0: <Google GOOGRSDT> acpi0: Power Button (fixed) acpi0: Sleep Button (fixed) cpu0: <ACPI CPU> on acpi0 atrtc0: <AT realtime clock> port 0x70-0x71,0x72-0x77 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0xb008-0xb00b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 isab0: <PCI-ISA bridge> at device 1.0 on pci0 isa0: <ISA bus> on isab0 pci0: <bridge> at device 1.3 (no driver attached) virtio_pci0: <VirtIO PCI (legacy) SCSI adapter> port 0xc000-0xc03f mem 0xfebfe000-0xfebfe07f irq 11 at device 3.0 on pci0 vtscsi0: <VirtIO SCSI Adapter> on virtio_pci0 virtio_pci1: <VirtIO PCI (legacy) Network adapter> port 0xc040-0xc07f mem 0xfebff000-0xfebff03f irq 11 at device 4.0 on pci0 vtnet0: <VirtIO Networking Adapter> on virtio_pci1 vtnet0: Ethernet address: 42:01:0a:8a:00:03 vtnet0: netmap queues/slots: TX 1/2048, RX 1/2048 000.000113 [ 450] vtnet_netmap_attach vtnet attached txq=1, txd=2048 rxq=1, rxd=2048 [... output deleted ...] attimer0: <AT timer> at port 0x40 on isa0 Timecounter "i8254" frequency 1193182 Hz quality 0 attimer0: Can't map interrupt. attimer0: non-PNP ISA device will be removed from GENERIC in FreeBSD 14. Timecounters tick every 10.000 msec [... output deleted ...] usb_needs_explore_all: no devclass Trying to mount root from ufs:/dev/gpt/rootfs [rw]... Root mount waiting for: CAM da0 at vtscsi0 bus 0 scbus0 target 1 lun 0 da0: <Google PersistentDisk 1> Fixed Direct Access SPC-4 SCSI device da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 30720MB (62914560 512 byte sectors) mountroot: waiting for device /dev/gpt/rootfs... [... output deleted ...] vtnet0: link state changed to UP
The CPU and memory are as we expected. The disk and other devices clearly show that we're running on a Google-specific virtualized cloud platform.
For a little more on the virtual hardware platform:
# lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 00:01.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 03) 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 00:03.0 Non-VGA unclassified device: Red Hat, Inc. Virtio SCSI 00:04.0 Ethernet controller: Red Hat, Inc. Virtio network device
As for the three Timecounter devices, we will need to
look deeper into those.
But first, let's look at the network environment.
The Ethernet interface is detected as
The /32 netmask seems wrong, but it works.
$ ifconfig vtnet0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1460 options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6> ether 42:01:0a:8a:00:03 inet6 fe80::4001:aff:fe8a:3%vtnet0 prefixlen 64 scopeid 0x1 inet 10.138.0.3 netmask 0xffffffff broadcast 10.138.0.3 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> lo: lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 1460 [... output deleted ...] $ netstat -nr Routing tables Internet: Destination Gateway Flags Netif Expire default 10.138.0.1 UGS vtnet0 10.138.0.1 link#1 UHS vtnet0 10.138.0.3 link#1 UH vtnet0 127.0.0.1 link#2 UH lo0 Internet6: Destination Gateway Flags Netif Expire ::/96 ::1 UGRS lo0 ::1 link#2 UHS lo0 ::ffff:0.0.0.0/96 ::1 UGRS lo0 fe80::/10 ::1 UGRS lo0 fe80::%vtnet0/64 link#1 U vtnet0 fe80::4001:aff:fe8a:3%vtnet0 link#1 UHS lo0 fe80::%lo0/64 link#2 U lo0 fe80::1%lo0 link#2 UHS lo0 ff02::/16 ::1 UGRS lo0 $ cat /etc/resolv.conf # Generated by resolvconf search c.cromwell-intl.internal google.internal nameserver 169.254.169.254
You're in a private network, a VPC or Virtual Private Cloud. It's something like the 10.138.0.0/24 network with just your server and a (virtual) router.
In an earlier step you reserved an external IP address. You assigned it to your VM and created the appropriate DNS records as shown on the previous page. The router does NAT for your VPC to that public address. Google will have PTR records in place, resolving to the googleusercontent.com domain.
The FreeBSD image comes with several packages added
to the basic install.
Make sure that you don't remove either
pkg, as you would need to back up and start over!
I know that because I initially wanted to use the Nginx
Installing that required many additional packages.
When I gave up on Nginx and decided to go with Apache,
I "cleaned up" by removing all added packages.
I did not realize that
pkg itself was an
OK, lesson learned. Delete the image, redeploy, and set up SSH again.
I found these 22 packages on the freshly deployed image:
I first installed all available updates for the existing packages.
Those brought along just 4 required libraries.
I also added
vim for personal preferences,
~/.vimrc file into place
for both my user account and
That required another 99 packages to satisfy dependencies!
Then I added the packages needed for Apache/PHP web service:
php73, and their dependencies.
Correcting Clock Problems
I soon noticed that there was a huge clock drift! Within one minute the system clock would be off by several seconds. A Google groups discussion gave me the needed hint.
# sysctl kern.timecounter.hardware kern.timecounter.hardware: TSC # sysctl kern.timecounter.choice kern.timecounter.choice: i8254(0) ACPI-fast(900) TSC(1000) dummy(-1000000) # sysctl kern.timecounter.hardware=ACPI-fast kern.timecounter.hardware: TSC -> ACPI-fast
That fixed my problem, so I added a line to
I verified that
/etc/rc.conf already contained
a line reading:
and I added a new line:
The second of those will resynchronize the system clock
at each boot using
and the first will keep the clock in sync while running
Proceed to the next step to see how to set up Apache and PHP.
Next step: Apache log and cache customization
Note: I changed from Apache to Nginx within the first year, with better performance and a more cautious configuration. See my page on running TLS 1.3 on Nginx for those details, then come back to these pages and the steps involved in installing dual TLS certificates.