Posts tagged gentoo


GHC 9.8 and Cabal 3.10.3 on Gentoo Linux

:: gentoo, haskell, programming language, system, tutorial

By: Maciej Barć

The official ::gentoo repository currently contains only GHC on version 9.2.8. To install newer GHC one has to either download/build themselves or use the ::haskell overlay (https://github.com/gentoo-haskell/gentoo-haskell).

Enable the ::haskell overlay

Enable:

1
eselect repository enable haskell

Sync:

1
2
3
emerge --sync haskell
egencache --update --repo haskell --jobs 12 --load 6
eix-update

Unmask needed packages

Add to /etc/portage/package.unmask/0000_hs.conf

1
2
3
4
5
6
7
8
<dev-lang/ghc-9.9

<dev-haskell/cabal-3.11
<dev-haskell/cabal-install-3.11
<dev-haskell/cabal-install-solver-3.11
<dev-haskell/cabal-syntax-3.11
<dev-haskell/text-2.2
<dev-haskell/parsec-3.1.18

Add to /etc/portage/package.accept_keywords/0000_hs.conf

1
2
3
app-admin/haskell-updater
dev-haskell/*
dev-lang/ghc

Install

1
emerge --ask --verbose ">=dev-lang/ghc-9.8" ">=dev-haskell/cabal-install-3.10"

Build of GHC 9.8 takes around ~2 hours on a 8-core laptop-grade CPU.

Bonus: masking packages from ::haskell

If you want to exclude a given version from the ::haskell overly from being installed/updated, then you can add a similar line(s) to /etc/portage/package.mask/0000_hs.conf:

1
app-emacs/haskell-mode::haskell

Do not split make.conf

:: gentoo, portage

By: Maciej Barć

I made a mistake when splitting my Portage make.conf file, having it as one file instead of a directly with many small files is a lot easier to maintain.

Portage allows users to split all of files inside /etc/portage such as make.conf, package.use, package.mask and other into groups of files contained in directories of the same name. This is very helpful when using automation to add some wanted configuration. But in case of make.conf it becomes a “form over function” issue.

I would also recommend to keep make.conf as simple as possible, without useless overrides and variable reassignment.

See also:

Bonus: config

And of course, this is my current /etc/portage/make.conf of my main dev machine:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
BINPKG_FORMAT="gpkg"
CCACHE_DIR="/var/cache/ccache"
EMERGE_WARNING_DELAY="0"
LC_MESSAGES="C"

PORTAGE_NICENESS="10"
PORTAGE_WORKDIR_MODE="0775"

PORTAGE_LOGDIR="${EPREFIX}/var/log/portage"
PORTAGE_ELOG_CLASSES="warn error log"
PORTAGE_ELOG_SYSTEM="save"

QUICKPKG_DEFAULT_OPTS="--include-config=y --umask=0003"
MAKEOPTS="--jobs=7 --load-average=6"

COMMON_FLAGS="
  -march=znver1 -O2 -falign-functions=32
  -fstack-clash-protection -fstack-protector-strong
  -fdiagnostics-color=always -frecord-gcc-switches -pipe"

ADAFLAGS="${COMMON_FLAGS}"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"

CARGO_TERM_VERBOSE="false"
RUSTFLAGS="-C opt-level=3 -C debuginfo=0"

LDFLAGS="${LDFLAGS} -Wl,--defsym=__gentoo_check_ldflags__=0"

L10N="en de pl"
VIDEO_CARDS="amdgpu radeon radeonsi"

CPU_FLAGS_X86="
  aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt
  sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"

EMERGE_DEFAULT_OPTS="
  --binpkg-changed-deps=y --binpkg-respect-use=y
  --nospinner --keep-going=y
  --jobs=3 --load-average=8"

GENTOO_MIRRORS="
  https://mirror.leaseweb.com/gentoo/
  https://gentoo.osuosl.org/
  https://distfiles.gentoo.org/"

FEATURES="
  userpriv usersandbox usersync
  downgrade-backup unmerge-backup binpkg-multi-instance buildsyspkg
  parallel-fetch parallel-install
  ccache
  -binpkg-logs -ebuild-locks"

USE="
  custom-cflags custom-optimization firmware initramfs vaapi vulkan
  -bindist -zeroconf"

.NET in Gentoo in 2023

:: dotnet, gentoo, packaging, portage, powershell

By: Maciej Barć

.NET ecosystem in Gentoo in year 2023

The Gentoo Dotnet project introduced better support for building .NET-based software using the nuget, dotnet-pkg-base and dotnet-pkg eclasses. This opened new opportunities of bringing new packages depending on .NET ecosystem to the official Gentoo ebuild repository and helping developers that use dotnet-sdk on Gentoo.

New software requiring .NET is constantly being added to the main Gentoo tree, among others that is:

  • PowerShell for Linux,
  • Denaro — finance application,
  • Ryujinx — NS emulator,
  • OpenRA — RTS engine for Command & Conquer, Red Alert and Dune2k,
  • Pinta — graphics program,
  • Pablodraw — Ansi, Ascii and RIPscrip art editor,
  • Dafny — verification-aware programming language
  • many packages aimed straight at developing .NET projects.

Dotnet project is also looking for new maintainers and users who are willing to help out here and there. Current state of .NET in Gentoo is very good but we can still do a lot better.

Special thanks to people who helped out

Portage Continuous Delivery

:: gentoo, linux, sysadmin, system

By: Maciej Barć

Portage as a CD system

This is a very simple way to use any system with Portage installed as a Continuous Delivery server.

I think for a testing environment this is a valid solution to consider.

Create a repository of software used in your organization

Those articles from the Gentoo Wiki describe how to create a custom ebuild repository (overlay) pretty well:

Set up your repo with eselect-repository

Install the my-org repository:

1
eselect repository add my-org git https://git.my-org.local/portage/my-org.git

Sync my-org:

1
emerge --sync my-org

Install live packages of a your software

First, enable live packages (keywordless) for your my-org repo:

1
echo '*/*::my-org' >> /etc/portage/package.accept_keywords/0000_repo_my-org.conf

Install some packages from my-org:

1
emerge -av "=mycategory/mysoftware-9999"

Install smart-live-rebuild

smart-live-rebuild can automatically update live software packages that use git as their source URL.

Set up cron to run smart-live-rebuild

Refresh your my-org repository every hour:

1
0 */1 * * * emerge --sync my-org

Refresh the main Gentoo tree every other 6th hour:

1
0 */6 * * * emerge --sync gentoo

Run smart-live-rebuild every other 3rd hour:

1
0 */3 * * * smart-live-rebuild

Restarting services after update

All-in-one script

You can either restart all services after successful update:

File: /opt/update.sh

1
2
3
4
5
6
7
8
#!/bin/sh

set -e

smart-live-rebuild

systemctl restart my-service-1.service
systemctl restart my-service-2.service

Crontab:

1
0 */3 * * * /opt/update.sh

Via ebuilds pkg_ functions

File: my-service-1.ebuild

1
2
3
pkg_postinst() {
    systemctl restart my-service-1.service
}

More about pkg_postinst:

Example Gentoo overlays

Genkernel in 2023

:: gentoo, kernel, linux, sysadmin, system, tutorial

By: Maciej Barć

I really wanted to look into the new kernel building solutions for Gentoo and maybe migrate to dracut, but last time I tried, ~1.5 years ago, the initreamfs was now working for me.

And now in 2023 I’m still running genkernel for my personal boxes as well as other servers running Gentoo.

I guess some short term solutions really become defined tools :P

So this is how I rebuild my kernel nowadays:

  1. Copy old config

    1
    2
    cd /usr/src
    cp linux-6.1.38-gentoo/.config linux-6.1.41-gentoo/
    
  2. Remove old kernel build directories

    1
    rm -r linux-6.1.31-gentoo
    
  3. Run initial preparation

    1
    ( eselect kernel set 1 && cd /usr/src/linux && make olddefconfig )
    
  4. Call genkernel

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    genkernel                                                       \
        --no-menuconfig                                             \
        --no-clean                                                  \
        --no-clear-cachedir                                         \
        --no-cleanup                                                \
        --no-mrproper                                               \
        --lvm                                                       \
        --luks                                                      \
        --mdadm                                                     \
        --nfs                                                       \
        --kernel-localversion="-$(hostname)-$(date '+%Y.%m.%d')"    \
        all
    
  5. Rebuild the modules

    If in your /etc/genkernel.conf you have MODULEREBUILD turned off, then also call emerge:

    1
    emerge -1 @module-rebuild
    

ELisp ebuilds good practices

:: elisp, emacs, gentoo, lisp, packaging

By: Maciej Barć

Check load path

Some Elisp package compilation failures are caused by not setting the loadpath correctly. It mostly happens when you compile source from a directory that is not the current working directory. For example:

1
elisp-compile elisp/*.el

In most cases you can cd or override the S variable to set it to location where ELisp source resides.

But in other cases you can append to load path the directory with source, see:

1
BYTECOMPFLAGS="${BYTECOMPFLAGS} -L elisp" elisp-compile elisp/*.el

Do not rename auto-generated autoload file

elisp-make-autoload-file allows to name the generated autoload file. For sake of easier debugging and writing Gentoo SITEFILEs, please do not rename the generated file.

The name of that file should always be ${PN}-autoloads.el.

Use new elisp-enable-tests function

elisp-enable-tests allows to set up IUSE, RESTRICT, BDEPEND and the test runner function for running tests with the specified test runner.

The 1st (test-runner) argument must be one of:

  • buttercup — for buttercup provided via app-emacs/buttercup,
  • ert-runner — for ert-runner provided via app-emacs/ert-runner,
  • ert — for ERT, the built-in GNU Emacs test utility.

The 2nd argument is the directory where test are located, the leftover arguments are passed to the selected test runner.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
EAPI=8

inherit elisp

# Other package settings ...

SITEFILE="50${PN}-gentoo.el"
DOCS=( README.md )

elisp-enable-tests buttercup test

Remove empty SITEFILEs

Recently a feature was added to elisp.eclass that will cause build process to generate the required SITEFILE with boilerplate code if it does not exist.

So if your SITEFILE looked like this:

1
(add-to-list 'load-path "@SITELISP@")

… then, you can just remove that file.

But remember to keep the SITEFILE variable inside your ebuild:

1
SITEFILE="50${PN}-gentoo.el"

Remove pkg.el files

The *-pkg.el files are useless to Gentoo distribution model of Emacs Lisp packages and should be removed. It is as simple as adding this line to a ebuild:

1
ELISP_REMOVE="${PN}-pkg.el"

Beware that some packages will try to find their ${PN}-pkg.el file, but in most cases this will show up in failing package tests.

Use official repository

It is tedious to repackage Elpa tarballs, so use the official upstream even if you have to snapshot a specific commit.

To snapshot GitHub repos you would generally use this code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# First check if we have the correct version to prevent
# autobumping package version without changing the commit.
[[ ${PV} == *_p20220325 ]] && COMMIT=65c496d3d1d1298345beb9845840067bffb2ffd8

# Use correct URL that supports snapshots.
SRC_URI="https://github.com/domtronn/${PN}/archive/${COMMIT}.tar.gz
    -> ${P}.tar.gz"

# Override the temporary build directory variable.
S="${WORKDIR}"/${PN}-${COMMIT}

Include live version support

We do not want to be worse than the Melpa unstable :D

So, why not allow the given package to be used live?

Even if you do not push the live package to the overlay, please include support for it.

1
2
3
4
5
6
7
8
if [[ ${PV} == *9999* ]] ; then
    inherit git-r3
    EGIT_REPO_URI="https://github.com/example/${PN}.git"
else
    SRC_URI="https://github.com/example/${PN}/archive/${PV}.tar.gz
        -> ${P}.tar.gz"
    KEYWORDS="~amd64 ~x86"
fi

Ask for tags

Git is good, git tags are good. In case if upstream does not tag their package or just forgets to, kindly ask them to create a git tag when bumping Emacs package versions.

Bubblewrap cross-architecture chroot

:: chroot, emulation, gentoo, linux, sandbox, system, tutorial, virtualization, vm

By: Maciej Barć

System preparation

Qemu

Emerge qemu with static-user USE enabled and your wanted architectures.

1
2
3
4
5
6
7
8
app-emulation/qemu      QEMU_SOFTMMU_TARGETS: aarch64 arm x86_64
app-emulation/qemu      QEMU_USER_TARGETS: aarch64 arm x86_64

app-emulation/qemu      static-user
dev-libs/glib           static-libs
sys-apps/attr           static-libs
sys-libs/zlib           static-libs
dev-libs/libpcre2       static-libs

OpenRC

Enable qemu-binfmt:

1
rc-update add qemu-binfmt default

Start qemu-binfmt:

1
rc-service qemu-binfmt start

Chrooting

  • select chroot location (eg /chroots/gentoo-arm64-musl-stable)
  • unpack the desired rootfs
  • create needed directories
    • mkdir -p /chroots/gentoo-arm64-musl-stable/var/cache/distfiles
  • execute bwrap
    • with last ro-bind mount the qemu emulator binary (eg qemu-aarch64)
    • execute the mounted emulator binary giving it a shell program (eg bash)

Chroot with bwrap:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
bwrap                                                       \
    --bind /chroots/gentoo-arm64-musl-stable /              \
    --dev /dev                                              \
    --proc /proc                                            \
    --perms 1777 --tmpfs /dev/shm                           \
    --tmpfs /run                                            \
    --ro-bind /etc/resolv.conf /etc/resolv.conf             \
    --bind /var/cache/distfiles /var/cache/distfiles        \
    --ro-bind /usr/bin/qemu-aarch64 /usr/bin/qemu-aarch64   \
    /usr/bin/qemu-aarch64 /bin/bash -l