Introducing easyinit

2Comments
Quirky Linux is intended mostly as a "full installation", occupying an entire partition, the same as most other distributions.

Unlike most major distributions, such as Debian and Ubuntu, Quirky does not have an initramfs. Instead, there is a kernel boot parameter "root=/dev/<partition>" which boots straight to the installation of Quirky. This makes for faster booting, also, Debian and Ubuntu have enormous initramfs files, consisting of files that are duplicated in the installed partition. This creates a larger .iso file to download.

Note, however, some of the variants of Debian and Ubuntu designed for specialised targets such as the Raspberry Pi, do not have a initramfs, or a very small one.

If Quirky needs to perform a filesystem check, resize the filesystem, or a system rollback (recovery, see the Snapshot Manager in the menu), /sbin/init, the first boot script in the installation detects this need, and creates a ramdisk with a minimal Linux installed into it, then performs a pivot_root into it. Then this minimal Linux is able to perform the required operations upon the partition in which Quirky is installed.

The main problem with this mechanism is the potential lack of robustness. The minimal Linux is created in the ramdisk by copying files from Quirky. However, if Quirky is badly broken, there is a possibility that the ramdisk will be also. Hence a rollback might not work.

To make this mechanism more robust, I have been on a mini-coding-marathon for the last couple of days. There is now a ready-made mini-Linux, called 'easyinit'. This is made of static applications that I compiled in Buildroot, with uClibc. The applications include busybox, coreutils, e2fsprogs, ntfs-3g, xdelta3 and gettext.

The main feature of easyinit is it is very small. Unlike the Debian/Ubuntu initramfs' that have kernel modules, libraries, shared-library executables, and a large array of support files.

easyinit resides in /boot in the Quirky installation, and /sbin/init just copies it into a ramdisk and performs a pivot_root into it.

Most significantly, /sbin/init has been reduced greatly in size. It is now a very small and simple script.

The functionality to perform filesystem check, resize, and rollback (recovery), is done within easyinit, and this is broken into small simple scripts.

In easyinit, there is the frontend, /init, that executes first. This may call /sbin/rollback or /sbin/fscheck.

All of this has been written, not yet tested.

Recovery for a totally broken system
The above mechanism will work if Quirky is able to at least get past the kernel, and the first boot script, /sbin/init, is able to run. Busybox in the Quirky installation is compiled statically, so the script should run even if glibc is broken.

However, if the system is so broken that even /init will not run, I plan to implement a fallback, "Plan B".

easyinit is a cpio archive, and can quite easily become an initramfs. So, in the case of the Raspberry Pi SD-card, if easyinit is copied into the first (vfat) partition, and the "root=" parameter changed to "initrd=easyinit", then easyinit will load into RAM and perform the required recovery.

This needs a bit more coding to implement, but it does mean that even a hopelessly broken installation can be recovered.
It does though, require that there be at least one snapshot of the system, taken by the Snapshot Manager.


T2 building small arm binaries

1 Comment
I posted yesterday about compiling 'xdelta3' statically in buildroot, native build on my Pi3:
http://barryk.org/news/?viewDetailed=00469

The 'xdelta3' binary is very small.

Now T2 is running on my Pi3, building the same selection of packages as used in Quirky April x86, as described here:
http://barryk.org/news/?viewDetailed=00470

The build is taking days. Quite literally, days. Right now, it is compiling some media libraries. I don't have a UPS, so hope the mains power supply doesn't hiccup.

I have just been comparing the sizes of executables compiled by T2 with those in my running Quirky 8.1.3.1 host, which is based upon Ubuntu armhf DEBs. The T2 binaries are smaller, no always, but in majority of cases.

In the T2 configuration, I chose the "cortex-a8" ARM CPU, and enabled thumb instructions.
When compiling, T2 uses these options for gcc:

-mcpu=cortex-a8 -mthumb -mthumb-interwork

I thought maybe the thumb instructions is the reason for the smaller binaries, however, I googled and found that Ubuntu and Debian armhf DEBs also are built with thumb-2 support.

So, I don't know why T2 is building smaller binaries.

Anyway, I am having to "cool my heels". Want to compile in buildroot, but can't until T2 has finished. Some other work also has to wait.


T2 on Pi3 now at Stage-3

2Comments
I reported yesterday that I was doing a native compile with T2 on my Raspberry Pi3:
http://barryk.org/news/?viewDetailed=00467

I am actually using an old version of T2, used back in 2015 to create Quirky April, for x86.

T2 Stage-0 builds a cross-toolchain, and at Stage-2 onwards there is a chrootable target build environment, as explained here:
https://www.t2-project.org/handbook/html/t2-book.html#t2.build-stages

Stages 0 and 1 were a rough ride. Mostly due to Ubuntu's header files conflicting with what is being compiled in the target environment.
I posted some questions and fixes to the T2 mail-list:
https://www.mail-archive.com/t2@t2-project.org/

However, there were lots of other nasty little hacks that I had to do. But finally got to Stage-2, and the going should be smoother, as the host headers won't interfere anymore.

I am typing on my Pi3 right now, even as it compiles. This Pi3 is fast enough to be a desktop replacement.
Right now, though, the CPU speed governor is set to "powersave", as I need to leave it unattended and would prefer to leave it running at a lower temperature.

I don't know if I want to do it, but if T2 does manage to compile most of the packages, I could build a Quirky April for the Pi. It would also be "T2 friendly", for any future T2 builds.


Xdelta3 compiled statically in Pi3

2Comments
Yesterday I tried to compile Xdelta3 statically against dietlibc, running Quirky on my Raspberry Pi3. Gave up.

Quirky, and the pup before, uses an old version of xdelta, "30p". This, and later versions, are described on the developer's website:
http://xdelta.org/

There is a "30q" which has fixes for Windows, then there was a major version jump. The old 30p works fine, so I will stay with it. It also looks simpler to compile.

Previously, I have compiled it in a Landley x86 and x86_64 uClibc chrootable filesystems, so I know that it does link against uClibc OK.

Well, I have buildroot. using uClibc, where I have already been compiling packages statically, so far, busybox, e2fsprogs and coreutils. The problem is, buildroot does not have xdelta3.

Fortunately, I found someone else has added xdelta3, version 30q, to buildroot:
https://github.com/wagle/addnas_source/tree/master/1470_Firmware_Source/buildroot-patches/packages/xdelta3

I modified that a bit, and put it into buildroot. I created folder package/xdelta3, with three files 'Config.in', 'xdelta3.hash' and 'xdelta3.mk'.

Config.in
config BR2_PACKAGE_XDELTA3

bool "xdelta3 binary file difference"
default y
depends on BR2_PACKAGE_UCLIBC
help
The xdelta3 patch tool. Allows incremental update of binary files.

http://xdelta.org/
<

Note that the indents are tab character for first indent, second indent is two space characters.

xdelta3.hash
sha1 988f8d8f884aee31b5a0de0a4067575cb452543f xdelta30p.tar.bz2
<

xdelta3.mk
#############################################################

#
# xdelta3
#
#############################################################
XDELTA3_VER:=0p
XDELTA3_SOURCE:=xdelta3$(XDELTA3_VER).tar.bz2
XDELTA3_SRC=xdelta3.c
XDELTA3_SITE:=http://distro.ibiblio.org/quirky/quirky6/sources/t2/april/
XDELTA3_DIR:=$(BUILD_DIR)/xdelta3$(XDELTA3_VER)
XDELTA3_CAT:=bzcat
XDELTA3_BINARY:=xdelta3
XDELTA3_TARGET_BINARY:=usr/sbin/xdelta3
XDELTA3_CC_OPTS:=-DXD3_DEBUG=0 \
-DXD3_USE_LARGEFILE64=1 \
-DREGRESSION_TEST=0 \
-DSECONDARY_DJW=1 \
-DSECONDARY_FGK=1 \
-DXD3_MAIN=1 \
-DXD3_POSIX=1


$(DL_DIR)/$(XDELTA3_SOURCE):
$(WGET) -P $(DL_DIR) $(XDELTA3_SITE)/$(XDELTA3_SOURCE)

xdelta3-source: $(DL_DIR)/$(XDELTA3_SOURCE)

#############################################################
#
# build xdelta3 for use on the target system
#
#############################################################
$(XDELTA3_DIR)/.unpacked: $(DL_DIR)/$(XDELTA3_SOURCE)
$(XDELTA3_CAT) $(DL_DIR)/$(XDELTA3_SOURCE) | tar -C $(BUILD_DIR) $(TAR_OPTIONS) -
touch $(XDELTA3_DIR)/.unpacked

$(XDELTA3_DIR)/.configured: $(XDELTA3_DIR)/.unpacked
touch $(XDELTA3_DIR)/.configured

$(XDELTA3_DIR)/$(XDELTA3_BINARY): $(XDELTA3_DIR)/.configured
${TARGET_CC} $(TARGET_CFLAGS) $(XDELTA3_CC_OPTS)\
${XDELTA3_DIR}/${XDELTA3_SRC} -o ${XDELTA3_DIR}/${XDELTA3_BINARY}

$(TARGET_DIR)/$(XDELTA3_TARGET_BINARY): $(XDELTA3_DIR)/$(XDELTA3_BINARY)
cp $(XDELTA3_DIR)/$(XDELTA3_BINARY) $(TARGET_DIR)/$(XDELTA3_TARGET_BINARY)


xdelta3: uclibc $(TARGET_DIR)/$(XDELTA3_TARGET_BINARY)

xdelta3-clean:
$(MAKE) -C $(XDELTA3_DIR) clean

xdelta3-dirclean:
rm -rf $(XDELTA3_DIR)

#############################################################
#
# Toplevel Makefile options
#
#############################################################
ifeq ($(strip $(BR2_PACKAGE_XDELTA3)),y)
TARGETS+=xdelta3
endif
<

Then, and entry has to be made in package/Config.in

Finally, compile it:
# make xdelta3
<


Heatsink for Pi3

2Comments
There has been some discussion on the Puppy Forum about heatsinks for the Raspberry Pi2 and Pi3. In particular, the aluminium case which also behaves as a heatsink for the CPU and RAM chips looks really great:
http://murga-linux.com/puppy/viewtopic.php?p=933743#933743

I bought some heatsinks from dx.com. Cheap, but tiny little things, only 5mm high. Don't think they will be much use at reducing the temperature, so I purchased this one from Adafruit, 15x15x15mm:
http://core-electronics.com.au/aluminium-heat-sink-for-raspberry-pi-3-15-x-15-x-15mm.html


And here it is on my Pi3, along with a smaller one from dx.com:


Nice and warm to the fingers. HardInfo is telling me 46.1 degrees C. But ambient is very mild, and just using it lightly right now, web browsing. Will need to stress test it!


Pages: [1] [2] [3] [4] [5] ...