Tag Archives: FreeBSD

logo-full-thumb

Tarsnap backup script

Tarsnap

Tarsnap is an advanced online-backup facility, entirely encrypted. The only copy of the keys used to encrypt and decrypt archives are in your own possession, so things that should be kept safe, are (in the current form) safe. Tarsnap makes extensive use of the Amazon EC2 and Amazon S3 for storage.

Tarsnap is originally written by the FreeBSD Security Officer Emiritus’ Colin Percival, on topics that he periodic gives talks about at various conferences. If you are able, you should seriously attend one of those talks

Script

Recently I rewrote a tarsnap backup script from Tim Bishop http://www.bishnet.net/tim/blog/2009/01/28/automating-tarsnap-backups/ to a more suitable script for us.

Tim backups his data via Tarsnap, all via the same way. That works well for him, but for our hosting company that is more tricky. We do not want to keep large amounts of data for our customers (which tend to change rapidly, for example emails that come in and go out and get deleted etc.). Instead we want to keep the minimal amount of data for these customers, and we want to offer them more advanced backup strategies for which we calculate an increased price (the minimal backup strategy is free).

After collaborating, we decided that next to the free strategy, we would like to offer a medium-term backup strategy, and a maximum-term backup strategy, where the former is a month of backups (7 weekdays, 4 weeks), and the latter is three months of backups (7 weekdays, 4 weeks, 3 months), so that going back in time is doable. If customers want to have a customized strategy, that would ofcourse be possible if we add that to the script.

Since we are keen on open source we would like to offer you the option to download the script, and if possible even enhance it more so that we can all benefit from it. Do note that we didn’t try to complicate the script, but instead keeping it as simple as possible. That means that we add more lines then likely needed, but it is very readable. One comment from Colin we got so far is that Tarsnap is capable of removing more files in one go (tarsnap -d -f -f ) and that is not yet implemented in the script. We will consider doing so.. ofcourse :-)

The script can be found here, tarsnap.script.

20131013
Updated the script with the update from Tim, this had been tested and works fine for us so far. Thanks Tim ! I shamelessly used the code in our code ;-)

logo-full-thumb

FreeBSD: jails, ezjail, pfSense

During the last couple of days I am intensively using ezjail to administer several jails on my machines. They are currently IPv6 only (internet-facing) and are used to build pfsense images to test locally (still setting this up, need to cross compile to i386 from amd64), offer a testjail to a collegue to work together on a Opsview implementation on FreeBSD, whether or not we are going to succeed in that, and I just installed a test environment for my webservices. They are all contained in their own little box, having IPv4 connectivity outgoing through NAT, and native IPv6 connectivity from my “Vendor” on an extra subnet that I obtained.

I like this, so I am probably going to setup some more services here and there to perform some magic for me that might need external access. I will also tie them together with LDAP and the like so that it’s an uniform base. At the moment I do not have additional ideas about moving production services towards jails as well though.

Thanks to FreeBSD this all is damned easy. You should try it, or poke me in case you want to know more! :)

FreeBSD RELENG_8 created

Yesterday Ken Smith created the RLENG_8, which is required before being able to release any 8.x version at all. This is the first step that will finally lead up to BETA3, after which RC1 is planned. Beta3 also marks the end of the “liberal” ‘ok we still allow some new features if they had previously been discussed’. No more new features will be inserted when BETA3 becomes live. I will try to make a nice list of what’s new for 8.0 the moment BETA3 is tagged. The release is one step closer again!

FreeBSD – clang

[lang_en]Ed Schouten started working on a FreeBSD – clang project. He did this by importing clang into a project repository and making sure that a ‘make buildkernel’ succeeds. (If I followed correctly). I believe this is one of the steps that he will be taking to get the LLVM compiler suitable for FreeBSD, and perhaps eventually replacing the GNU licensed gcc compiler. That (like his TTY work) would be one hell of a job! Please poke ed@FreeBSD.org if you are interested in helping![/lang_en]

[lang_nl]Ed Schouten is begonnen met het werken aan het FreeBSD – clang project. Hij is hiermee begonnen door clang in een project repository te importeren, en ervoor zorg te dragen dat een `make buildkernel’ slaagt (Als ik het goed gevolgd had). Ik denk dat dit een belangrijke stap is om te zorgen dat de LLVM compiler geschikt wordt voor FreeBSD, welke wellicht ooit de GNU gcc compiler kan vervangen. Net als zijn TTY werk zou dit een onwijs gaaf iets zijn. Neem contact op met ed@FreeBSD.org als je geinteresseerd bent om te helpen![/lang_nl]

How popular is FreeBSD 7.1 and should I proceed with my info?

So i wrote some articles recently for 7.1 and the release-cycle around it to try and provide as much possible information without telling you all things that we as a project cannot make true etc. I see that my “how to upgrade to 7.1″ article is being read a lot, in fact it is one of the most popular items in my entire article-range. Which made me wonder about a few things, which I hope that you (my readers) can answer:

*) Is FreeBSD 7.1 long awaited and popular?
*) Aren’t there enough documents that provide this straight to the point information? (Do this and this and you have 7.1 running)
*) Should I proceed with these kind of articles for the future release-cycle’s?
*) How is 7.1 being received? Do you like it? Do you hate it?

Reasons I want to know this, is whether I should bring in my attention to this area now which I might be doing a ‘difference’ again, and whether it really helps the community or not. And last but not least ofcourse, we tried to work hard to bring you the best release possible, but I personally cannot entirely judge how the community received our work. I think we did great, but I cannot speak for you :)

If you read this, and are willing to help, please respond to this article :-)

Thanks,
Remko

FreeBSD 7.1 popular?

After writing my little blog on how to upgrade to 7.1 I got several people telling me that the freebsd-update servers were slow, irresponsive etc. From the information that Colin send to FreeBSD-stable it seems that a lot of people were really waiting for 7.1 to come out, and that they all wanted to upgrade. Colin had made improvements to the system so that it can better deal with the load.

The good thing from this is that a lot of people ARE interested in the latest version, as well as that they use the easy to use application “freebsd-update”. If you run a -RELEASE version, you should use this to easily and quickly upgrade to a later revision.

Keep those bits rolling!

How to upgrade to FreeBSD 7.1

With 7.1 around the corner it might be handy to tell what you need to do in order to upgrade.

First of all you need to understand that I am in no way responsible for the actions you are doing on your system. I can give an advise, but if there is one typo somewhere it could trash your system. That is the case now, and it will forever be as long as there are computers :)

That said: Do you know what kind of user you are?

Did you install via the CD and never heared about CVSup? SVN? and things like that?

Then FreeBSD-update (/usr/sbin/freebsd-update) is the thing for you!
It’s the most simple way to upgrade from a previous -RELEASE (or Security branch) to the current one. Do note that -STABLE or -RC’s are not taken into account and cannot be used to upgrade to 7.1-RELEASE.

In order for you to upgrade please use : /usr/sbin/freebsd-update -r 7.1-RELEASE upgrade && /usr/sbin/freebsd-update -r 7.1-RELEASE install

After this the system will tell you:

Kernel updates have been installed. Please reboot and run
“/usr/sbin/freebsd-update install” again to finish installing updates.

(Thanks richard for the information):

Voila, you will be running 7.1 now!

For people using CVSup: replace the RELENG_7_0 part with RELENG_7_1 (it’s ok if there are other things behind it, as long as you change the zero to a one (0 -> 1)).

Rebuild your kernel as you probably did many times before (cd /usr/src && make buildworld && make buildkernel && make installkernel [reboot in single user mode] mergemaster -p && make installworld && make delete-old && mergemaster [reboot] make delete-old-libs (be careful that ports and things do not depend on older versions. I have been bitten by an account that used the bash shell, with libreadline, which was deleted by make delete-old-libs.. no more bash shell for me!)

If you are doing CVS checkouts you should probably do something with -r RELENG_7_1 or RELENG_7_1_RELEASE to get things going, I would advise the first one since patches will be installed there as well, the _RELEASE branch is static. Then do the tricks like above as well.

If you are a subversion user you should checkout the 7.1 release (all subversion users know how to do this) and perform a bit of magic.