How to successfully build packages for WD My Cloud from source

The WD My Cloud comes with a Debian wheezy system on it. However WD customized that in a way that it is hardly possible (if not impossible at all) to install new packages using the standard "apt-get install <package name>" way, especially when you update the device firmware to version 4.x or later. The latest firmware, in fact, uses a modified Debian system with 64K sized memory pages: if you install a package using "apt-get install <package name>" from the standard repositories, almost certainly it won't run and produce just a laconic "Killed" output.
So, to install new packages on the My Cloud you need to build them from source on another system, copy the obtained deb packages on the My Cloud and install.
WD provides a GPL source package of the latest firmware on their website, which contains a build environment to perform this operation. This is totally at the end-user own risk, since no support at all is provided by WD and installing 3rd party software may void warrany.
An additional problem is that the build environment provided by WD has a "minor" problem that causes the building process of most packages to fail because of a GCC compiler segmentation fault.
This guide shows how to deal with this problem and, in general, how to create a build environment to easily do the task.
Some Linux knowledge is needed.
NOTE 1: as of now, this procedure is surely needed to build packages for the 4.x firmware; however, by experience, I find out that it's a useful procedure even if you are sticking with the 3.x firmware; so the guide explains how to build packages for both firmware versions
NOTE 2: in principle, it should be possible to build packages directly on the My Cloud, instead of using an external system; however I don't think it's a good idea to try this, because of many reasons, including: the My Cloud system is surely slower than a regular PC to compile packages; you would need to install all the development tools on the My Cloud itself (and this may require in turn to build them on another system first...)
Step 1: prepare the build system (required only once)
The build environment must be a Linux system, either Debian-based or Ubuntu-based. I personally suggest to create a virtual machine with such a system in it. This guide will take this route.Download and install VirtualBox (https://www.virtualbox.org/) on your phisical system (either Linux, Windows or Mac). Start VirtualBox and create a new virtual machine for a Debian 64-bit. The default settings suggested by VirtualBox regarding memory, disk size and VM configuration should be ok to start.
After the VM is ready, download a Debian Wheezy 64-bit ISO image; I suggest the netinst image, which is the smaller one. Right now, Debian 7.8.0 is available and the direct links are:

注意，强烈推荐用最新的Debian 8.3系统来进行编译，就目前的编译经验来讲，Debian 8.3所使用的Qemu修正了大量的BUG，尤其是编译期间出现的大量的"段错误(Segment fault)",7.8版本上完全无法通过编译的软件，在8.3上轻松编译通过。

Boot the VM with the downloaded ISO mounted in and install just the Debian base system (nothing else is required). Refer to VirtualBox and Debian websites for documentation on how to perform these operations.

Once the guest VM is installed, we need to make just a couple little tunings. First of all, Debian Wheezy comes with qemu-user-static package version 1.1.2. Qemu is an environment needed to emulate an actual ARM system on another platform, like the AMD64 platform our build system consists of. It's a good idea to update Qemu to version 2.x from the wheezy-backports repository. To do this, start the build system and login. Then:

对于国内的用户，使用官方源可能无法更新，建议如下设置

如果使用现在最新的Debian 8.3版本，则不需要继续在backports中安装qemu了，执行如下命令即可。

This should also install binfmt-support, which is another packages needed by the build environment. If this is not the case, also type:

Now, let's prepare the actual build environment. Let's create a folder in the /root directory of the build system and download the WD My Cloud 4.x firmware source package from WD website.

In case the link changes, refer to WD My Cloud support page to find the new one: http://support.wdc.com/product/download.asp?groupid=904&lang=en

I then suggest to create different folders for different build scenarios. These are the possibilities:

  • the target system may be firmware 3.x (4k) or firmware 4.x (64k)
  • the source package base may be wheezy (Debian stable) or jessie (Debian testing); wheezy contains older packages, but that should run happily in My Cloud (which has a Wheezy in it!), while jessie contains newer packages that might also work and provide updated versions of many applications; I would personally recommend to build packages from wheezy, unless you absolutely need a newer version that is only in jessie

I don't recommend to mix things, so I would create different folders for any different combination. You're free to create just the one you are interested in, so among the following commands type fhe first ones, then only the block of commands of the combination(s) you're interested in, then the last command:

In this way, in every folder will be created a build.sh script that passes the right parameters to the WD provided script, requiring only the name of the package to build. This would work straight away if there weren't the problem with qemu I mentioned in the beginning, so another step is required to finish the prepare phase. Again, only type commands for the scenario(s) you're interested in:

64k-wheezy:

64k-jessie:

4k-wheezy:

4k-jessie:

The meaning of the above is the following: prepare an emulated ARM system and replace the qemu-arm-static binary provided by the bootstrap with the recent one we've installed in our actual build system.

Ignore any errors produced by setup.sh: that script is really buggy and many things it tries to do seem to be useless, unless we apply the mentioned qemu fix.

As a final step, I would recommend to edit the sources file list within the armhf build subsystem in order to be able to build packages that are in any of the distribution repositories. To do this, type the following:

by replacing <scenario> with the desired one (64k-wheezy, 64k-jessie, 4k-wheezy, 4k-jessie); the nano editor will open, then replace the contents of the existing file with the following:

For 64k-wheezy and 4k-wheezy:

For 64k-jessie and 4k-jessie:

In case of wheezy, uncomment the last two lines if you want to build package versions from wheezy-backports. I don't know however if additional changes to the build.sh script (or better to the WD provided one) are needed to instruct apt to download and build the source from the backports repository. I don't have tried it yet. Anyway, I would recommend to leave those two lines commented unless you actually need something from the backports repository.

Save the file by hitting Ctrl+X, Y, Enter.

Now you're ready to build your first package!

如果使用现在最新的Debian 8.3版本，需要把主机的域名解析的配置文件，复制到编译的对应目录，否则会造成无法解析域名(当然，也可以自行编辑一个正确的resolv.conf文件，指定域名解析服务器地址）。

Optional additional step (not strictly required) for wheezy scenarios

You may also want to use an updated C++ compiler to build packages in the wheezy scenarios. Debian Wheezy provides g++ package 4.6, but 4.7 is also available. With the following commands you can install the new version and then switch from one to the other using update-alternatives:

After these commands, the default C++ compiler will be version 4.7. You can then switch to the old version by typing:

Or use

to get an interactive prompt.

Step 2: build your package
It's time to build your package. Let's build htop for instance, a nice replacement for top command. The example will build it for the scenario 64k-wheezy (i.e: packages suitable for 4.x firmware, built from the version of htop provided by Debian Wheezy). Start your build system, login, then:

The built package will be placed into /root/wdmc-build/<scenario>/build/root and will consist of one or more .deb files (you'll also find other files and folder, just ignore them and consider only .deb files). Sometimes building a package will build other packages, too: it depends on how package sources are organized. For instance, if you build transmission-daemon, you'll also get the deb packages for the GUI packages (like transmission-gtk or transmission-qt): you can simply throw away those additional debs if you don't need them.

Once the deb files are ready, copy them from the build system to your My Cloud (to achieve this, if you built a VirtualBox VM you may use a shared folder to exchange data with the physical host system). Then, login via SSH to your My Cloud and type:

where <path-to-built-deb-file> is the path where you stored the deb file you just built; if you copied it to the standard Public shared folder, the path will be /shares/Public/.

If you're lucky, you're done. Otherwise, the /installation command may prompt you something like this:

This simply means that the package you're trying to install depends on another package (in this case: transmission-daemon requires libcurl3-gnutls) which is not yet installed. You then need to build in turn this required package, with the same procedure, and install it before the package you originally intended to install. Repeat the steps for all the missing dependencies that dpkg reports. Sometimes, to fix circular dependencies (packages depend on each other), once you've built all the necessary packages you may install them all at once with: dpkg -i <path-to-built-debs>/*.deb, so that dpkg will resolve dependencies automatically.

Final notes

  1. building a package may require a LOT of time, just be patient...
  2. you may periodically clean /root/wdmc-build/<scenario>/build/root folder after you've built your packages, to free up space; in fact, building may require a considerable amount of (temporary) disk space
  3. once installed with dpkg, packages contents are unpacked to the My Cloud system partitions. There should be enough disk space available to not worry about disk space, however almost certainly a firmware upgrade will delete all of your installed packages; I would suggest to create a shared folder using the web UI, named "System", protected by password and accessible only to admin users; inside it you may put your built deb files, organized as you like. In case of a firmware upgrade, you may simply re-install the packages from there using dpkg -i (one at a time, or all of them at once using wildcards, etc.); however please consider the following notes
  4. a firmware upgrade might change the architecture of the provided Debian system and might require to rebuild all packages... this is exactly what happened with the upgrade from 3.x to 4.x firmware, where WD decided to rebuild the whole system using 64k sized memory pages rather than the standard 4k sized ones... I don't think WD is so sadistic to do that again and again, but who knows? I suggest to read carefully the release notes of the new firmware, have a look to the updated GPL source package (if they make it available on their website) and try to install a few small packages after firmware upgrade just to check that everything is ok; installing all in a rush may have catastrophic effects, especially when a package you had built on your own requires to update one of the WD provided ones... if it won't work, it will almost certainly break something of the WD built-in features, causing a device failure in the worst case... again, you are on your own, you get no support for this
  5. to restore the whole functionality of third-party applications after a firmware upgrade, not only you may need to reinstall the deb packages, but you will also need to restore any custom application configuration; if you leave default settings, usually applications save their settings in /etc or in /root, which are both on the system partitions of WD and might be deleted and recreated by the firmware update process; you may choose to backup your application configurations or change things so that they are linked to a shared folder (for instance the aforementioned System shared folder) where they won't be deleted; in this way, after the firmware upgrade you may just need to restore your backups or your symlinks; howerver, the exact procedure may be different for each package and is above the scope of this guide

From the above, two things should be quite clear: disable the automatic firmware upgrade in the web UI and use all of this at your own risk.

目前已知的包编译问题：

  • runit-2.1.1包编译的时候，会提示失败

原因为其中的一个检查脚本对于 chroot重定向之后的目录检查存在问题，解决方法为找到 “/wdmc-build/64k-wheezy/build/root/runit-2.1.1/runit-2.1.1/src/check-local”，注释掉里面的代码，修改

为

  • 编译 gcc-4.6的时候，会提示失败

解决方法为，手动执行所有的命令，不要让他执行创建子进程命令。

注意，不要编译 gcc-4.7 版本，因为会导致 libstdc++6升级，而libstdc++6升级之后会导致系统工作不正常。

另外，上面的编译完成之后,安装GCC-4.6的时候，会提示libgomp1 这个包不存在，貌似ARM下面到 Gcc-4.6 的时候还不能很好的支持OpenMP,网上的其他编译GCC 4.6 ARM的都是禁用掉了OpenMP的编译。

修改里面的

为

然后重新编译即可。

  • 编译Subversion的时候提示失败

这个时候是不能编译成功的，必须用一个非root 的用户执行命令，执行如下命令,增加一个普通用户user,并且安装fakeroot,然后用user的身份登录

此时如果编译还是编译不通过的，主要是JAVAHL部分不能编译通过，ARM下面用的JAVA解析器是gcj-jdk,这个解析器貌似根本不能运行起来，最后会卡在如下的部分，尝试过切换到Open-jdk,可惜也不能编译过去，因此只能去掉这部分的编译

这时候需要修改源代码的debian/rules文件,去掉JAVAHL部分的编译

修改

为

然后修改

为

然后继续执行上面的命令，并且切换用户到user

以上为Subversion-1.6 Wheezy 版本的编译，如果要编译 Subversion-1.8 Jessie版本的话，还需要如下配置

1.用user用户登录

2.删除Subversion-1.8 目录

3.重新以低权限生成目录，如果不如此操作，会提示各种各样的权限不足。

4.返回root，并安装缺少的依赖库

5.修改debian/rules，禁止测试安装包，目前启用测试会失败。

修改为

6.重新返回 user，执行编译。

  • 编译 GCC 4.9

很遗憾的是Gcj,Ecj模块在ARM下面即使到了GCC-4.9都不能正常运行，运行的时候，直接卡死在运行界面上面，既不失败，也不返回，就是卡住比如安装 ecj-gcj 后在Shell 直接运行 ecj-gcj，连版本号都不能显示，就是卡住。

因此只能在ARM下面要求GCC不支持JAVA的编译，编译JAVA还是用OpenJDK 好了。

打开规则配置文件

修改

为

也就是在最后面增加

去掉JAVA模块，然后重新编译即可。

  • 编译最新的 aria2

编译最新的Jessie版本的 aria2，如果使用 Jessie 的编译环境编译，则编译出来的程序依赖了gcc-4.9 导致libc6的升级，一旦 libc6升级，则系统立即崩溃，因此编译方法为把64k-Jessie 下面的文件夹拷贝到 64k-wheezy目录下面然后编译即可。

  • 编译 PHP5

编译PHP5的时候，会出现卡住在 stop mysqld 的问题，此时应该新开一个shell,然后手工杀掉mysql服务，貌似是编译程序没办法停止mysql,只能是手工处理,杀掉后，编译进程就可以正常继续了。

  • 编译 update-inetd

编译update-inetd-4.43的时候，总是失败，卡住在

因此需要注释掉 debconf-updatepo

修改

为

然后重新编译即可。

  • 编译 git-2.1.4

编译最新的Jessie版本的 git-2.1.4,(git版本低于1.8的服务器存在一个当大量文件被pull到服务器会出现内存峰值导致内存不足而提交失败的BUG），如果使用 Jessie 的编译环境编译，则编译出来的程序依赖了gcc-4.9 导致libc6的升级，一旦 libc6升级，则系统立即崩溃，因此编译方法为把64k-Jessie 下面的文件夹拷贝到 64k-wheezy目录下面然后编译即可。

需要注意的是libpcre3-dev的开发包安装语句

否则会提示如下错误：

  • 编译 git-lfs

  • 编译 amule

amule默认编译的时候是包含GUI界面的，但是WD MyCloud是用不到界面的，实际上GTK+部分目前也没办法编译通过。
首先，执行一次编译，让脚本自动把需要的依赖安装配置完成，方便我们的后续处理

amule编译完成后，把依赖的wxWidgets2.8等依赖库编译完成后，只能安装命令行版本的amule，安装命令如下：

  • 编译 nut-2.7.2

ups不间断电源网络服务

如果发生如下错误：

执行如下命令修复后重新编译：

如果发生如下错误：

执行如下命令修复后重新编译：

如果发生如下错误：

执行如下命令修复后重新编译：

如果发生如下错误：

执行如下命令修复后重新编译：

如果发生如下错误：

执行如下命令修复后重新编译：

如果发生如下错误：

执行如下命令修复后重新编译：

如果发生如下错误：

执行如下命令修复后重新编译：

如果发生如下错误：

执行如下命令修复后重新编译：

如果发生如下错误：

执行如下命令修复后重新编译：

如果发生如下错误：

执行如下命令修复后重新编译：

  • 编译 libnspr4-4.12

  • 编译 libnss3-3.26

  • 编译 tcp-wrappers-7.6

  • 编译 apcupsd-3.14.12

ups不间断电源网络服务（需要先编译[tcp-wrappers-7.6])

  • 编译 e2fsprogs-1.42.13

参考链接

GUIDE-Building-packages-for-the-new-firmware-someone-tried-it

发布者

默默

码农