Ideas and notes

Wishlist

Wed, 18 Aug 2010 00:07:44 GMT

gmirror

Mon, 16 Aug 2010 03:44:00 GMT

Here are the steps for setting up RAID-1 with gmirror. I am assuming that you have two identical IDE hard drives setup in a master/slave configuration on ad0 and ad1 and that you have FreeBSD-6.2-RELEASE installed on drive ad0. It's important to note that if you're using SATA or SCSI drives then your device names will differ from my example.

Step 1: Use sysctl to allow the mounted disk to be modified
Step 2: Create the mirror on ad0 with gmirror
Step 3: Modify /boot/loader.conf to load gmirror on startup
Step 4: Modify /etc/fstab for gmirror device names
Step 6: Use gmirror insert command to add the 2nd drive for mirroring
Step 7: Check the mirroring status with gmirror status

[Allow the mounted disk (ad0) to be modified by gmirror]

This step allows the gmirror utility to modify your drive while it is mounted and is a one time operation during gmirror setup. Login in as the root user and issue the sysctl command:

sysctl kern.geom.debugflags=16

[Create the mirror on ad0 with gmirror]

This command modifies the drive label metadata on your hard disk so that it is recognized as a gmirror RAID disk. I'm labeling the raid set gm0 but you can give it any label you like as long as you're consistant. It's important to note that this label name becomes your new device file name used in /etc/fstab.

gmirror label -v -b round-robin gm0 /dev/ad0

Metadata value stored on /dev/ad0

[Modify /boot/loader.conf to load gmirror on startup]

Using your favorite editor, ee or vi for example, modify your loader.conf file so that the gmirror module automatically loads at startup.

vi /boot/loader.conf

Add the following text to the /boot/loader.conf file and save it:
geom_mirror_load="YES"

[Modify /etc/fstab for gmirror device names]

In this step you are modifying the /etc/fstab file so that it will mount the gmirror device gm0 rather than ad0 that the system was previously using. Here is what your old fstab should look like:

old /etc/fstab device file

/dev/ad0s1b none ...
/dev/ad0s1a / ...
/dev/ad0s1e /tmp ...
/dev/ad0s1f /usr ...
/dev/ad0s1d /var ...

Again, use your favorite editor, ee or vi to modify the /etc/fstab file. Here is what your modified /etc/fstab file should look like. Notice that I have added the subdirectory, "mirror" to the path and replaced ad0 with gm0 for all the device names (including the swap file)

newly modified /etc/fstab device file

/dev/mirror/gm0s1b none ...
/dev/mirror/gm0s1a / ...
/dev/mirror/gm0s1e /tmp ...
/dev/mirror/gm0s1f /usr ...
/dev/mirror/gm0s1d /var …

That's all there is to the gmirror configuration! Now we just need to reboot, let the system come up, and add our ad1 drive to the system for mirroring using the the gmirror insert command. A word of caution- Double check your work. If you mis-typed ANYTHING in the /boot/loder.conf or /etc/fstab file then the system will NOT come up!

[Use the gmirror insert command to add the 2nd drive for mirroring]

Assuming the system comes up fine, and I'm sure it has (grin), then all we have to do to complete our project is tell gmirror to mirror our drives together. We will use the gmirror insert command to do this. We will insert our second hard disk (ad1) into our gm0 RAID. Here is the command to do it.

gmirror insert gm0 /dev/ad1

GEOM_MIRROR: Device gm0: provider ad1 detected
GEOM_MIRROR: Device gm0: rebuilding provider ad1

The systm will now be busy for awhile mirroring the two drives. The time it takes to complete the task depends on your system hardware and drive sizes. However, you can check the status of your rebuild with the gmirror status command.

[Check the mirroring status with gmirror status]

gmirror status

Name Status Components
mirror/gm0 DEGRADED ad0
ad1 (18%)

Note that the system has detected the array to be "DEGRADED" and is automatically rebuilding the drive array. Also note that it is 18% complete. When it reaches 100% the gmirror status reads "COMPLETE" and you're done! Isn't it crazy how simple this is?

My observations when recovering a RAID array

Here's what I did to test the gmirror RAID system. First, I installed FreeBSD and created a mirror set with gmirror (using two IDE drives). After the RAID system was up and running, I simulated a drive failure by shutting the system down and removing the IDE master drive (ad0) from the system. Then I re-installed the IDE slave drive (ad1) as the primary master and rebooted the system. To my surprise and glee the system came right up with no errors and all data intact!!

After my FreeBSD system came up fine, I used another PC to wipe the old ad0 master hard disk. I then placed it back in the system, as a slave, and rebooted. A gmirror insert command returned GEOM_MIRROR: Cannot add disk ad1 to gm0 (error=17) so I tried the command: gmirror insert gm0 /dev/ad1 in an attempt to get the drive back into the mirror. Unfortunately I received another error so I re-read the gmirror man page for more info. I decided to disconnect the second drive (the slave),reboot, login and issue the gmirror forget command:

gmirror forget gm0 /dev/ad1

The system returned an error about not finding the device /dev/ad1. I shutdown, connected the drive again and rebooted. I issued the same command again:

gmirror forget gm0 /dev/ad1

and then the

gmirror insert gm0 /dev/ad1

command again. This time it inserted the drive and began rebuilding the array. Hurray!

As always, please refer to the man pages for help. The man pages for commands used in this article are:

man gmirror
man vi
man loader.conf
man fstab
man sysctl

Logitech_720_Notes

Sun, 09 May 2010 21:25:00 GMT

Programming the Logitech 720 for use with XBMC

1. You must UNPAIR any Apple Remotes you have already paired with OS X in System Preferences -> Security -> General.

2. In XBMC's System Settings for Apple Remote, set it to Multi-Remote.

3. If you are interested in the ability to launch XBMC using the Menu button on your Harmony One, enable the Always On setting. Please note that this will prevent the use of the Harmony One OR any Apple Remote by any other applications, including Front Row, Sofa Control, Remote Buddy, Plex, etc.

4. We'll assume you've installed the Logitech Harmony Remote software and have already used it to configure other devices.

5. Create a new device of type "Media Center PC", brand Plex, and type "Plex Player" into the box.

6. Create a new Activity using this Plex Player device and any other devices you need (TVs, amps, etc).

7. Rename this Activity something like "Watch XBMC"

8. Customize the buttons for this Activity as shown in these screenshots.

HTPC

Sun, 09 May 2010 21:00:00 GMT

Goal: Ability to play music, and watch movies, DVDs, or TV shows all from one interface, with one remote (to rule them all)

Hardware
Mac Mini
1GB RAM
80GB Disk
DVI -> Component for SD TV
1Gbit link connects mini and NAS
Logitech 720 Remote Notes
Apple BT keyboard

Software
XBMC: Used to read files from my NAS
iTunes: Easy creation and management of playlists
DoubleTwist:Syncing media to my Nexus One

GeekTool

Wed, 21 Apr 2010 17:20:00 GMT

Element	Shell Command	Font	Location	Size	Update Period (sec)
The day	date +%A	Helvetica, Regular, 27point with drop shadow	10x, 727y	200w x 40h	60
The month	date +%B	Helvetica, Regular, 36point with drop shadow	10x, 750y	200w x 50h	60
The day of the month	date +%d	Helvetica, Regular, 64point with drop shadow	210x, 720y	100w x 80h	60
The time	date "+%I:%M"	Helvetica, Bold, 72point with drop shadow	20x, 780y	200w x 80h	10
am/pm	date +%p	Helvetica, Regular, 48point with drop shadow	210x, 785y	100w x 70h	10
The temperature	bin/weather.rb -t	Helvetica, Bold, 18point	20x, 855y	70w x 25h	300
The weather	bin/weather.rb -d \| fmt -w 40	Helvetica, Regular, 10point	90x, 855y	210w x 38h	300
When the weather was updated	bin/weather.rb -w	Helvetica, Regular, 8point	35x, 878y	64w x 25h	300

#!/usr/bin/ruby
require 'rss/2.0'
require 'open-uri'
require 'optparse'
require 'fileutils'
 
summary = false
forecast = false
temperature = false
today = false
weatherdate = false
 
opts = OptionParser.new
opts.on("-s") { |val| summary = true }
opts.on("-f") { |val| forecast = true }
opts.on("-t") { |val| temperature = true }
opts.on("-d") { |val| today = true }
opts.on("-w") { |val| weatherdate = true }
opts.parse!
 
# Check if another process is downloading the weather and block until it's done
while File.file?('/tmp/weather.rb.tmp.lck')
  sleep(0.1)
end
 
# Download the weather if it's out of date
if !File.file?("/tmp/weather.rb.tmp") || ((Time.now - File.mtime("/tmp/weather.rb.tmp")) > 1800)
  FileUtils.touch('/tmp/weather.rb.tmp.lck')
  `curl --silent -m 30 "http://rss.wunderground.com/auto/rss_full/<your-state>/<your-city>.xml?units=english" > /tmp/weather.rb.tmp`
  if File.size("/tmp/weather.rb.tmp") == 0
    FileUtils.rm("/tmp/weather.rb.tmp")
  end
  FileUtils.rm('/tmp/weather.rb.tmp.lck')
end
 
# Parse out the weather results
File.open('/tmp/weather.rb.tmp') do |f|
  response = f.read
  result = RSS::Parser.parse(response, false)
  result.items.each_with_index do |item, i|
    puts "#{item.title.gsub(/ - .*/, '')}"  if summary == true and i == 0
    puts "#{item.description.strip}\n\n" if forecast == true and i > 0
    puts "#{item.title.gsub(/Current Conditions : /, '').gsub(/,.*/, '')}" if temperature == true and i == 0
    puts "#{item.description.gsub(/Today - /,'').gsub(/Tonight - /,'').gsub(/This Afternoon - /,'').gsub(/[\r\n\t]/, '')}" if today == true and i == 1
    hour = item.pubDate.hour()
    if hour < 12
      ampm = "AM"
    else
      ampm = "PM"
    end
    if hour == 0
      hour = 12
    end
    if hour > 12
      hour = hour - 12
    end
    puts "#{item.pubDate.mon()}/#{item.pubDate.day()} #{hour}:#{'%02d' % item.pubDate.min()} #{ampm}" if weatherdate == true and i == 0
  end
end

SnowLeopardNotes

Fri, 09 Apr 2010 17:26:00 GMT

VerboseBooting

Set printing options that aren't in the gui here

SingleUser Repairs

SingleUser Repairs

Fri, 09 Apr 2010 17:26:00 GMT

Start by loading the Directory Services 'launchctl /System/Library/LaunchDaemons/com.apple.DirectoryServices.plist'

Then we can use dscl to manipulate things.

Resetting_the_SMU

Sat, 27 Feb 2010 04:40:00 GMT

Apple Support KB
Power Mac G5 (Late 2005)
To reset the SMU on a Power Mac G5 (Late 2005) computer, either use the steps listed above for the Power Mac G5 (Late 2004) or:

Turn off the computer by selecting Shut Down from the Apple menu or by holding the power button until the computer turns off.
Open and remove both the metallic outer door and the inner plastic air deflector.
Remove the fan assembly immediately to the left of the processor module.
Press the SMU reset button on the logic board.
Replace the fan assembly, air deflector, and outer door.
Turn on the computer.
The SMU reset button is located underneath the lower bank of system memory slots, as shown below:

Unlike earlier models of Power Mac G5 or Power Mac G4 computers, the Power Mac G5 (Late 2004) and (Late 2005) models do not have a PMU. This functionality has been replaced by the SMU.

Personal

Sat, 27 Feb 2010 04:38:00 GMT

Hardware
Software
Photography
Bicycles
Wishlist
Nerdly Inspiration
TiddlyWiki
BayLISA

Hardware

Sat, 27 Feb 2010 04:38:00 GMT

Storage
Cisco
HTPC
Apple_G5

Apple_G5

Sat, 27 Feb 2010 04:38:00 GMT

Resetting the SMU

Software

Sat, 27 Feb 2010 04:37:00 GMT

FreeBSD
OSX
Xen
FreeNAS
FreeNAS_Vinnie
PGP

LeopardNotes

Fri, 05 Feb 2010 21:21:00 GMT

Enable an Apple Mac OS X machine as a syslog server
Here is a small howto that describes how your Mac OS X machine can also receive logs from remote devices such as an Apple Airport Extreme. There are some howto's available online, but I guess that somethings have changed in 10.5, none seem to work perfectly.

Change syslogd configuration

echo "local0.notice /var/log/airport.log" >> /etc/syslog.conf

Touch the logfile

touch /var/log/airport.log

Change syslogd startup procedure
At the end of the file, uncomment the part to accept remote logging.

cat /System/Library/LaunchDaemons/com.apple.syslogd.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.apple.syslogd</string>
    <key>OnDemand</key>
    <false/>
    <key>ProgramArguments</key>
    <array>
<!--
Un-comment the following lines to run syslogd with a sandbox profile.
Sandbox profiles restrict processes from performing unauthorized
operations; so it may be necessary to update the profile
(/usr/share/sandbox/syslogd.sb) if any changes are made to the syslog
configuration (/etc/syslog.conf).
-->
<!--
<string>/usr/bin/sandbox-exec</string>
<string>-f</string>
<string>/usr/share/sandbox/syslogd.sb</string>
-->
<string>/usr/sbin/syslogd</string>
    </array>
<key>MachServices</key>
<dict>
<key>com.apple.system.logger</key>
<true/>
</dict>
<key>Sockets</key>
<dict>
<key>AppleSystemLogger</key>
<dict>
<key>SockPathName</key>
<string>/var/run/asl_input</string>
<key>SockPathMode</key>
<integer>438</integer>
</dict>
<key>BSDSystemLogger</key>
<dict>
<key>SockPathName</key>
<string>/var/run/syslog</string>
<key>SockType</key>
<string>dgram</string>
<key>SockPathMode</key>
<integer>438</integer>
</dict>
<!--
Un-comment the following lines to enable the network syslog protocol listener.
-->
<key>NetworkListener</key>
<dict>
<key>SockServiceName</key>
<string>syslog</string>
<key>SockType</key>
<string>dgram</string>
</dict>
</dict>
</dict>
</plist>

Restart syslogd

# launchctl unload /System/Library/LaunchDaemons/com.apple.syslogd.plist
# launchctl load /System/Library/LaunchDaemons/com.apple.syslogd.plist

Open the firewall
Go the the System Preferences, click Security, open the Firewall tab and click the +. Select the file /usr/bin/syslog.
If you are unable to select the /usr directory, try this hack by opening a terminal and typing:

$ cd
$ ln -s /usr/bin
$ ln -s /usr/sbin

Now you can select the file (in your home directory) ./bin/syslog and ./sbin/syslogd

Blackberry and Leopard
Blackberry I have
Pocket Mac

Getting Time Machine to work on an Unsupported NAS

Some of us may own Network Attached Storage (NAS) devices, and/or may feel that an Apple Time Capsule is too expensive, and hence would rather buy a seperate router, NAS and Hard Disk Drives. This guide will help you to enable Time Machine on any NAS, allowing you to have the functionality of a Time Capsule, with hardware of your own choosing.

1. Prepare your NAS. Make sure it is fully functioning. This mini-guide assumes you already have one that is functioning well, and you know how to manage it.
2. It would be preferable to assign your NAS a Static IP address. Check your router and NAS documentation for guidelines on how to do this.
3. In OS X, open up Terminal and enter the following command to enable Time Machine to work with your NAS: defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1
4. Log out of OS X and Log in again. You do not need to reboot your machine.
5. Time Machine saves backups in a very specific format, and we will use a sparsebundle to get this done. The sparsebundle name consists of your Computername_MACaddress.sparsebundle.
6. Find the MAC address of the machine’s internal Ethernet port with ifconfig en0 | grep ether | sed s/:g | sed s/ether in Terminal. This will return one line of output, which is the MAC address for the Ethernet port, which will be a string of 14 hexadecimal characters (letters and numbers, for example, 001ec4b8f9b3).   Even if the network backups will be done using a different port (e.g. AirPort: usually en1), the system will use the MAC address of en0 as part of the system identifier.
7. Make a new “sparsebundle” on a local disk (NOT the Time Machine disk!). This sparsebundle is a virtual filesystem image which we’ll copy to the NAS, and Time Machine will then access it remotely (that way Time Machine’s not limited by the filesystem features of whatever NAS it’s using: all the funky stuff happens within the sparsebundle). By default sparsebundles can keep growing until they fill up the NAS, but in this scenario we keep other things on the NAS as well as backups, and we’re going to limit the sparsebundle size to 70 GB.Enter the following command in Terminal:
sudo hdiutil create -size 70g -type SPARSEBUNDLE -nospotlight -volname “Backup of My Mac” -fs “Case-sensitive Journaled HFS+” -verbose ./Computername_MACaddress. This will create a 70GB sparse-bundle as a case-sensitive, journaled HFS+ without spotlight indexing. Substitute variables in red for values you need. Computername_MACaddress may be in the form of “DansComputer_001ec4b8f9b3
8. Log into your NAS and create a user Account (Eg, “TimeMachine”). You may want to assign disk space quotas for this user if you need to.
9. On your NAS, create a shared folder (Eg, “TimeMachine”). Add the user account you created in step 8 above, and grant that user Read+Write privileges.
10. In Finder, select Go>Connect to Server. For the server address, type smb:TimeMachine@ip-address-of-nas/TimeMachine. Click Connect and when prompted, enter the password and save in your keychain.
11. Now copy your locally created sparsebundle into the TimeMachine share. If you did not change the folder when you entered command prompt, you may find this sparsebundle in your Home folder (/Users/yourusername).
12. On your NAS, check to see that the file is copied under the correct user account.
13. Open up Time Machine, select your “TimeMachine”-folder as the desired destination, and the backup will start.
14. Tip: For the first backup it is recommended to connect via LAN, as you will be able to get a throughput of 10-30MB/sec.
15. If your NAS allows for it, you may now hide your TimeMachine folder so they do not show up over the network share list.

AddressBook iSyncing

Step 1

Open up Terminal.app, but if you aren’t confortable with the terminal you might want to follow this tip instead. Once you have Terminal open, navigate to ~/Library/Preferences/. We will need to edit the com.apple.iPod.plist file located here. First make a backup of this file then take a look at its contents. If the file is a binary file you will need to execute the following command:
plutil -convert xml1 com.apple.iPod.plist

Now you should be able to edit the file with your favorite text editor (TextWrangler, vim, etc). Change whatever number is below ‘Family ID’ to 10001 as shown below.

Save your changes and run the following command to convert the file back to a binary:
plutil -convert binary1 com.apple.iPod.plist

Note: You may want to duplicate the iPod entry to prevent this from being disabled by future iPod syncs. To do so copy everything contained within the and below,
000B27111236C6CB

…

and paste just below the original entry. Then simply change the number slightly to prevent overwriting.
Step 2

Open Address Book and go to the Preferences-General pane and check the box to enable Synchronize with Gmail. You’ll be prompted to enter your Gmail account information (see below).
If you don’t have any iSync devices such as mobile phones or PDAs then you will also need to enable Yahoo syncing for this to work, see this guide for more information.

Step 3

Now open iSync.app and go to the Preferences panel. Check the box to show status in the menu bar (see below). Quit iSync.app.

Step 4

Make sure iSync.app is not running and go the the iSync menu bar icon and select ‘Sync Now’. You will be prompted to replace or merge with your Google Gmail Address Book.

Sometimes getting “Sync Now” to show up can be tricky, my only suggestion right now is to reboot and try again.

Repeat

Simply repeat steps 1-4 on any other Macs where you want your address book to be available.

To change the Google sync client type, open a terminal window and type (one line)
sudo defaults write /System/Library/PrivateFrameworks/GoogleContactSync.framework/Resources/ClientDescription Type 'server'

Then type the following one line command to restore the permissions on the plist file:
sudo chmod 644 /System/Library/PrivateFrameworks/GoogleContactSync.framework/Resources/ClientDescription.plist

Stop auto-switching in Spaces

Problem: Spaces is great and all, but it forces me to sort by applications instead of tasks. For example, if I select {application} from the Dock, then it automatically switches me to the virtual desktop space in which I first started {application}. However, what I really wanted was to start another instance of {application} running in my current virtual desktop space. Assigning {application} to all desktops in the Spaces preferences isn't the solution, because then Spaces just drags around all {application} windows onto all desktop spaces.

Solution: Issue the following command in a Terminal or xterm (X11) window:

defaults write com.apple.dock workspaces-auto-swoosh -bool NO

Restart the Dock using the following command:

killall Dock

Spaces will now allow you to start multiple instances of an application in separate desktop spaces. Note that implementing this setting can result in confusing behavior if you have used the Spaces preferences to assign a specific application to a specific desktop spaces (although assigning appplications to all desktop spaces appears to be fine). Repeat with “NO” replaced by “YES” to restore the default behavior. The file modified by defaults in this case is ~/Library/Preferences/com.apple.Dock.plist.

OS Version Compatibility: Leopard (10.5.2+) only.

AutoDelete

Mon, 14 Dec 2009 19:27:00 GMT

(Referenced here)

This refinement of Sebastian's method takes advantage of features in MySQL 4.x and later to encapsulate the deletions in a transaction so that data integrity is maintained even while new mail items are being received.

Start by creating the index on maia_mail.score as before:

CREATE INDEX maia_mail_idx_score ON maia_mail (score);

Next, create the view but use a temporary table to store it so that we can safely delete from the actual maia_mail table (since it's referenced in the view itself):

CREATE ALGORITHM = TEMPTABLE VIEW maia.HighScore AS
       SELECT maia_mail.id AS id
              FROM maia_mail
              WHERE maia_mail.score >= 20;

Now copy the following SQL code to a file (e.g. maia-autodelete.sql):

START TRANSACTION WITH CONSISTENT SNAPSHOT;
   DELETE FROM maia_mail_recipients
          WHERE mail_id IN (SELECT * FROM maia.HighScore);
   DELETE FROM maia_sa_rules_triggered
          WHERE mail_id IN (SELECT * FROM maia.HighScore);
   DELETE FROM maia_viruses_detected
          WHERE mail_id IN (SELECT * FROM maia.HighScore);
   DELETE FROM maia_banned_attachments_found
          WHERE mail_id IN (SELECT * FROM maia.HighScore);
   DELETE FROM maia_mail
          WHERE id IN (SELECT * FROM maia.HighScore);
COMMIT;

To execute the script, just feed it to MySQL:

mysql -u root -p maia < maia-autodelete.sql

Robert's original post can be found here.

MaiaMailGuard

Mon, 14 Dec 2009 19:23:00 GMT

Auto-Deleting entries that score >= X

Consulting

Mon, 14 Dec 2009 19:22:00 GMT

Security Planning
FreeBSD
Solaris
Sendmail
Linksys
Bouncing mail
DNS testing (random ports)
Maia Mailguard

BayLISA

Thu, 12 Nov 2009 06:59:00 GMT

Current Postmaster stuff
Sendmail pulled down and built - DHW $HOME
customized majordomo
majordomo aliases are in seperate file
most lists are closed
blw is configured so that members of blw or baylisa can post
milter-regex
perl script in ~dhw looks at /var/log/syslog and summarizes things

Tiddly_Formatting

Mon, 09 Nov 2009 01:48:00 GMT

TiddlyWiki

Mon, 09 Nov 2009 01:44:00 GMT

Tiddly Formatting

ZFS

Sun, 08 Nov 2009 23:16:00 GMT

Recommended reading:

FreeBSD's ZFS wiki
Sun's OpenSolaris ZFS

How to install FreeBSD 7.0 under ZFS

ZFS is an exciting new file system developed by Sun and recently ported to FreeBSD. Many people are excited by the possibilities of ZFS (including us) as it promises to simplify a great many things. It offers:

* increased reliability through checksums, multiple copies of data and self-healing RAID
* elimination of that dreaded “oh, if only I’d made the /var partition larger” feeling. Partitions can now be resized at any time and in fact can each be allocated up to the full size of the storage media.
* built-in compression and encryption
* built-in NFS file sharing
* clean, easy to use toolset for creating storage pools, volumes and much more
* snapshots and rollbacks for backups

Read more about some of the powerful commands here
Installing ZFS on FreeBSD

Since it is early days for ZFS on BSD, the installer doesn’t yet support ZFS natively. So there are a few tricks to getting it up and running. Also, you cannot boot directly from a ZFS partition since adding that functionality to the boot loader in FreeBSD is a huge undertaking.
Step One: installing FreeBSD

* Boot up on the FreeBSD Current 7.0 CD
* Choose Country and Locale
* Go Custom install
* Partition fdisk with Auto option (one slice, whole disk)
* Disklabel with options

A: 512Mb UFS2 /
B: swap
D: rest of disk

To create D you’ll need to enter any mount point you want and then use the M option to clear it. This ensures that it will not mount or be created as a file system.

Distribution choose Minimal install
Media Select CD/DVD

System will install to the small 512Mb UFS root partition you created.
Step Two: creating the ZFS pool

* Once installation is complete, remove CD and reboot into our new FreeBSD system.
* Boot into FreeBSD partition and select “4” for single user mode.
* Hit ENTER to accept /bin/sh shell.

mount -w /

Now create a Disk pool using the D label we prepared during the install. In this example we have a SATA disk at ad4.

zpool create tank /dev/ad4s1d

Firstly stop ZFS from creating default mountpoints for shares

zfs set mountpoint=none tank

Create some extra/common mountpoints

zfs create tank/root
zfs create tank/usr
zfs create tank/var
zfs create tank/tmp

Now set the mountpoints of the shares

zfs set mountpoint=/tank tank/root
zfs set mountpoint=/tank/usr tank/usr
zfs set mountpoint=/tank/var tank/var
zfs set mountpoint=/tank/tmp tank/tmp

Have a look to what we have done

df -h

zfs list

Beautiful isn’t it? :)

Edit /etc/rc.conf and enable ZFS

echo 'zfs_enable="YES"' >> /etc/rc.conf

Now copy the UFS bootable slice to the ZFS mountpoint. This gives us a workable FreeBSD installation under ZFS.

find -x / | cpio -pmd /tank

(you can ignore any errors here if you get them)
Step Three: solving the ZFS boot problem

The problem which remains is that FreeBSD will not be able to boot directly into ZFS since the bootloader doesn’t know anything about ZFS. So a little trick is that we put the kernel onto /boot which lives on the UFS partition. This gets the system running to the point where ZFS can be mounted and the rest of the boot proceeds.

Remove the /tank/boot just copied over from the UFS system:

rm -rf /tank/boot

Now make the directory in which our UFS partition will be mounted. This will be useful later on when we want to update the contents of that slice from a running system. We also need to make sure that when the ZFS is booting up it can see the UFS bootdir.

mkdir /tank/bootdir
cd /tank
ln -s bootdir/boot boot

Now we tell the loader on the UFS slice to load and boot from the contents of the ZFS volume:

echo 'zfs_load="YES"' >> /boot/loader.conf
echo 'vfs.root.mountfrom="zfs:tank/root"' >> /boot/loader.conf

Edit /tank/etc/fstab so our UFS slice is mounted in the right location for when ZFS boots.

/dev/ad4s1a /bootdir ufs rw 1 1

When we reboot in a minute, we want the ZFS tank to mount in /var /usr /tmp and/ and not within the /tank location it is now. So set the true mountpoints:

zfs set mountpoint=/tmp tank/tmp
zfs set mountpoint=/usr tank/usr
zfs set mountpoint=/var tank/var

Set root mount point to ’legacy’ so ZFS won’t try to mount it automatically. It should already have been mounted by the loader:

cd /
zfs set mountpoint=legacy tank/root

All Done!

Reboot and login as root. df -h and zfs list and you will see everything.

df -h

Filesystem Size Used Avail Capacity Mounted on
tank/root 282G 454M 281G 0% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/ad4s1a 496M 317M 139M 69% /bootdir
tank/tmp 281G 1.1M 281G 0% /tmp
tank/usr 287G 5.7G 281G 2% /usr
tank/var 281G 76M 281G 0% /var

zfs list

NAME USED AVAIL REFER MOUNTPOINT
tank 6.27G 281G 454M none
tank/root 16.2M 51.3G 16.2M legacy
tank/tmp 1.10M 281G 1.10M /tmp
tank/usr 5.75G 281G 5.75G /usr
tank/var 75.5M 281G 75.5M /var

All is done. :) Basic install completed. At this point you might want to install a more complete version of FreeBSD, install the ports system or anything else you normally do.
Creating a Mirror

That was pretty exciting.

Now let’s say that we have some important data on that drive and we are pretty keen not to lose it. So, let’s add another drive as a mirror under ZFS. In some ways this is even superior to a hardware RAID setup since ZFS is able to monitor the checksums on the disk and automatically detect which of the two drives has corrupted a block of data, transparently using the other and repairing if necessary.

* Install the HDD and boot up on the system.
* Login as root
* Run sysinstall
* Enter Custom Install
* Partition fdisk with Auto option (one slice, who disk) and press ’w’ to write changes
* Label with options

A: 512Mb /
B: swap
D: rest of disk empty (for zfs later)

and once again, press ’w’ to save changes and state ’YES’ to the question it asks you about applying now. You’ll need to make this identical to the labels on the first drive.

Exit sysinstall.

Check out what we have:

zpool status

Add the 2nd drive to our mirror (our second drive comes up as ad6)

zpool attach tank ad4s1d ad6s1d
zpool status

and it will now show the two HDD’s in a mirror

pool: tank
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror ONLINE 0 0 0
ad4s1d ONLINE 0 0 0
ad6s1d ONLINE 0 0 0

errors: No known data errors

Now we have a mirror, but if we lose the first drive we will need the contents of the UFS slice copied to the second drive in order to reboot.

newfs /dev/ad6s1a
mkdir /mnt/bootdir
mount /dev/ad6s1a /mnt/bootdir
find -x /bootdir | cpio -pmd /mnt

This gives us another ready to use system on the 2nd HDD in case we have a failure on the 1st HDD with all the necessary zfs tools to move forward.
/boot/Loader.conf tweaks

As you know, ZFS within Freebsd is and Experimental system and there are still some bugs to iron out, if the system has so much disk access you crash, you can kernel panic with “kmem_suballoc” you can fix this by placing the following in /boot/loader.conf.

vm.kmem_size_max=”512M”
vm.kmem_size=”512M”
vfs.zfs.zil_disable=1

The amount for the kmem size is up to you but if you make sit too large you will kernel panic on boot up.
Disaster recovery

(Maybe necessary if you have just completed the before mentioned tweaks and rebooted)

With the way we have setup our system if the ZFS fails to boot or function for whatever reason we can still boot off the original minimal install at any time. To do this do the following:

* Boot up
* Choose #6 (Escape to Loader Prompt) at the boot menu.
* Suppress the mounting of the ZFS volumes like this and disable the zfs module:

unset vfs.root.mountfrom
disable-module zfs

* Suppress of kmem sizes

unset vm.kmem_size
unset vm.kmem_size_max

* Single user mode and boot

set boot_single
boot

* Login

and edit the /boot/loader.conf as necessary

df -h
zpool list

You will notice that it has not picked up the ZFS drives or the pool. Never fear we can import it!

zpool import -d /boot/zfs

This loads the zpool data so the libraries can find the information it needs, now we need to import our pool using an alternate root.

zpool import -f -R /tank tank

And check out that it has worked.

df -h
zpool list

Useful Commands

Create a mirror

zpool create -m /usr/local/www/ mirror ad4s1g ad6s1g

Show available pool/resources

zfs list

Show Status of Disk/pool

zfs status

Changing Mount points for created resources

zfs set mountpoint=/

Data Limiting

zfs set quota=10g testpool/testfs/dir1

(logically limits space)

zfs set reservation=20g testpool/testfs/dir2

(logically preallocates space)

Unloading a kernel and choosing another one (in the boot loader mode)

Upgrading (such as 7.x -> 8.x)

Follow steps in /usr/src/UPDATING
buildworld/kernel
reboot single user
mount -o rw tank/root /
mount -t zfs tank/usr /usr
mount -t zfs tank/var /var
cd /usr/src
mergemaster -U -u 0022 -p
rm -fr /usr/include.old && date && mv /usr/include{,.old} && date
make NO_FSCHG=yes installworld
make delete-old
mergemaster -U -u 0022 -i
reboot

unload kernel
boot/boot/kernel.old/kernel

Replacing a failed drive

zpool replace tank
zpool status

Ok...
the scripts are at:
http://dist.k1.com.br/scripts/baselist_amd64
http://dist.k1.com.br/scripts/baselist_i386
http://dist.k1.com.br/scripts/makebootdisk
http://dist.k1.com.br/scripts/zfsetup

install these scripts on /root

makebootdisk:
formats the disk (or usb stick) at da0,da1...) make a bsdlabel on it
and using the baselist file, copies the running system files into the
USB
it will work on FreeBSD versions greater 7.0
this way the usb is bootable, have a filesystem on it.
the same root password...
you can fix the files /etc/rc.conf, /boot/loader.conf in the usb
filesystem in
order for it to boot from your kernel.
remeber to check for an "a" partition on your usb stick

the script needs to have access to install bash (pkg_add -r bash) so it
needs internet of a package repository with bash in it.

Once boot from your usb stick, you can do the same procedure to
transport the running system to another disk
if you intend to make a zfs running filesytem on the target disk (hd)
make the disklabel (bsdlabel) this way
a: 1gb 16 unused
b: 4gb * swap
d: * * unused
that is:
a partition 1gb at offset 16
b swap partition 4gb after partion A
d: the rest of the disk (this will hold the zpool).

the makebootdisk will install a running system on A (about 300mb...)

ZFSETUP
is a script that, when boot from the hd created with makebootdisk
moves the running system (booted from "a" partition) to the the zfspool
created, in the "d" partition mentioned above..
it edits the loader.conf in order to boot on zfs.. in order to boot
from

After that, you are running on ZFS...