Discussion:
Use of disklabel, MBR and GPT
(too old to reply)
Rocky Hotas
5 years ago
Permalink
Hello!
After partitioning a new disk with MBR, creating two primary partitions,
I was trying to define a disklabel in each of them. Quite soon I
realized it was not possible. After an IRC conversation, which was very
instructive yesterday, here are some considerations.

An important premise is that disklabel, MBR and GPT are data structures
intended to describe a whole disk, not a single partition: internally,
they then divide the disk into partitions. These single partitions can
not host themselves a disklabel, an MBR or a GPT to further subdivide
them, because they are not an entire disk.

Maybe this is trivial, but during the installation of NetBSD a
disklabel is actually nested inside a primary MBR partition. So, I would
like to share some further clarification.

Assume that you have this example MBR:

1. NTFS, Windows system partition
2. NTFS, Windows data
3. (empty)
4. (empty)

(for simplicity, I will not consider the case of extended partitions,
but it would be similar).

Assume that, when installing NetBSD with sysinst(8),

<http://www.netbsd.org/docs/guide/en/chap-exinst.html>

partition 3. is chosen for NetBSD. sysinst(8) does not change MBR in
this case (thus preserving primary partitions 1. and 2.).

There are several other options:

- sysinst(8) may create a brand new MBR (warning: this will destroy all
the existing data on the disk), with primary partitions defined by the
user, who can then choose which of them will be used for NetBSD;
- through sysinst(8) the user may also modify an existing MBR, and in
this case it's his responsibility to preserve the partitions dedicated
to other OSs and to manage only the free space in the disk for NetBSD;
- sysinst(8) also offers the chance to `Use the entire disk': in this
case, it creates anyway a MBR, with a primary partition which extends
to the whole disk, used for NetBSD.

As you may note, the use of MBR is never abandoned. This is done for two
reasons:

- to grant the compatibility with PC/DOS (and other OSs like Windows
this way can still use the same disk);
- to grant a regular boot on PC/IBM hardware (where, without a valid
MBR, the BIOS could not even try to search a bootloader on the disk).

Note that the above behaviour is about amd64; for other ports
(https://www.netbsd.org/ports/) it may change.

Whatever choices the user makes, in the end there will be an MBR primary
partition dedicated exclusively to NetBSD. Inside it, sysinst(8) installs
disklabel. disklabel(8) allows to further divide this NetBSD-dedicated
MBR partition, originating disklabel partitions `a' (which is generally
used for the NetBSD root directory /), `b' (generally used for swap), `e'
(it may be /home, for example), `f' (for example, /usr), `g' (for
example, /var), etc.; in amd64, partitions `c' and `d' are reserved, to
represent respectively the NetBSD MBR primary partition and the whole
disk, as shown here:

<http://www.netbsd.org/docs/guide/en/chap-inst.html#chap-inst-installdoc>

It may seem that this disklabel, nested into an MBR primary partition,
is used to further divide internally this partition: so, for another MBR
primary partition on the same disk, there may be another disklabel which
can be used the same way. This is wrong.

Disklabel is an alternative, and not an integration, to MBR.

In the case of an amd64 NetBSD installation inside a PC/DOS disk, the
disklabel is placed inside the MBR primary partition dedicated to NetBSD,
in the second block (and the first block is used for the NetBSD
bootloader). This is done to make sure that the disklabel data structure
is preserved and not altered or deleted by the action of other OSs: in
fact, the MBR primary partition dedicated to NetBSD is not used by DOS
or similar OSs, but it is recognized by them.
Without this issue, disklabel (whose location is flexible) could be
placed in the second sector of the disk (while MBR is in the first), not
the second sector of a specific partition.

When the machine is booted with NetBSD, disklabel is the only way NetBSD
looks at the disk, the *entire* disk; MBR in this case is not considered
at all. (Obviously, during the installation, a reference to the NetBSD
bootloader is placed in the boot sector of MBR, so that NetBSD can be
booted). Disklabel provides exactly all the information MBR provides,
but in a way NetBSD can understand. It provides the full size of the
disk (disklabel partition `d' for amd64), the size and location of the
usable portion of the disk (disklabel partition `c', for example, which
represents in amd64 the NetBSD primary partition), and the structure of
this portion of the disk (disklabel custom partitions `a', `b', `e',
`f', `g', etc.). Thinking about MBR, it actually provides the same
information, but in a way the other OSs can understand: the full size
of the disk, the size and location of the usable portion(s) of the disk
and their structure (all the primary partitions whose type is
DOS-compatible).

Disklabel and MBR are then just two alternative ways to represent the
same reality, the whole disk. The former is used by NetBSD and the
latter is used by DOS-compatible OSs like Windows, for the same disk.
The location of disklabel (nested inside an MBR primary partition) is
just a consequence of possible compatibility issues with other OSs.
But both MBR and disklabel are meant to be unique inside a disk (and
in systems where PC/IBM compatibility issues are absent, they should
not even coexist: disklabel only is used; on the other hand, in a
Windows PC, only MBR is used). A disk can not host more than one MBR,
or more than one disklabel.
My initial intention of creating multiple disklabels, one for each
MBR primary partition, was wrong.

Figure 2.1 in

<http://www.netbsd.org/docs/guide/en/chap-inst.html#chap-inst-installdoc>

well describes the way disklabel (and so, NetBSD) looks at the whole
disk. Refer to the blue labels, on the right of the image.
The same Figure also well describes the way MBR (and so, a
DOS-compatible OS) looks at the whole disk: refer to the red labels, in
the center of the image.

The fact that NetBSD only uses disklabel for his awareness of the disk
is also proved by the /dev directory: for each disk, for example wd0,
only the disklabel partitions are available: wd0a, wd0b, etc. (along
with the corresponding raw partitions rdw0a, rwd0b, etc.). There is no
way to reference the other MBR primary partitions (so, installing a
disklabel on them would be impossible, other than uncorrect), because
from the NetBSD (and disklabel) perspective, they simply do not exist:
it only exists a portion of the disk which is not used by any
disklabel partition (and where actually are placed the eventual other
MBR primary partitions).

All the above considerations are assuming that a single NetBSD OS is
installed in a disk. They may change is two or multiple NetBSD systems
are installed in the same disk, but this is a possible conflicting and
confusing scenario, which I would discourage.

This is different on FreeBSD, where the MBR primary partitions are
accessible for each disk: for example, /dev/ada0s1 refers to the first
MBR primary partition of disk ada0 (`s' means `slice', another way MBR
primary partitions are called).
In FreeBSD, /dev also has ada0s1a, ada0s1b, etc., if ada0s1 has a UFS
filesystem). I don't know if FreeBSD also manages in a different way
the coexistence of MBR and disklabel.

GPT is used for the same purpose as MBR and disklabel: it describes the
structure of an entire disk. There's however a difference in the way
NetBSD integrates with it: while MBR is an alien system for NetBSD, GPT
is supported. So, for example, if a new disk is added to an existing
NetBSD system, and this disk is partitioned with GPT, there's no need of
a disklabel. For each new partition, GPT in NetBSD automatically creates
a wedge (dk0, dk1, etc.). Wedges are the equivalent of a partition, not
a disk, so a disklabel can not be placed on them (also, /dev has no
`dk0a', `dk0b', etc., but only `dk0', `dk1', etc.). A new filesystem
(FFSv2, for example) can be directly created on a wedge. I don't know
if a disklabel is installed when GPT is used to partition a NetBSD
system disk (the one hosting the root `/' directory), but I don't think
so.

GPT is available to partition a system disk when performing a UEFI
installation:

<https://wiki.netbsd.org/Installation_on_UEFI_systems/>

but IIRC a traditional BIOS installation does not offer GPT.

As well, if a new disk is added to an existing NetBSD system (so, it is
not a boot disk), it may be partitioned using only disklabel, not MBR or
GPT. Avoiding in amd64 the reserved partitions `c' and `d', the other
letters may be arbitrary related to any data contents (with any
mountpoint).

These observations would not have been possible without the help of the
users in the #netbsd channel on Freenode IRC. I would like to thank them
so much. If any of them wants to integrate these notes and/or correct
something, is welcome.

Thank you again!

Rocky

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Martin Husemann
5 years ago
Permalink
I have a few clarifications...
Post by Rocky Hotas
Maybe this is trivial, but during the installation of NetBSD a
disklabel is actually nested inside a primary MBR partition.
This is true for some architectures (like amd64), but others do use plain
disklabel on a disk without any MBR. This also works on amd64 when you
do not need to boot from the disk.
Post by Rocky Hotas
As you may note, the use of MBR is never abandoned. This is done for two
- to grant the compatibility with PC/DOS (and other OSs like Windows
this way can still use the same disk);
- to grant a regular boot on PC/IBM hardware (where, without a valid
MBR, the BIOS could not even try to search a bootloader on the disk).
Yes and no - newer sysinst versions allow to kill the whole disk content
(including the MBR and whatver partitions exists), and start from scratch
(as if the disk had been totaly wiped before). Use with care - this kills
all data on the disk.
Post by Rocky Hotas
Whatever choices the user makes, in the end there will be an MBR primary
partition dedicated exclusively to NetBSD.
If you pick MBR partitioning, yes. If you select GPT, it is different
(as you mentioned below).
Post by Rocky Hotas
It may seem that this disklabel, nested into an MBR primary partition,
is used to further divide internally this partition: so, for another MBR
primary partition on the same disk, there may be another disklabel which
can be used the same way. This is wrong.
Right. There is (a maximum of) one disklabel per disk. On architectures
supporting MBR the kernel will use the one from the first MBR NetBSD
partition (if it exists), or the global one - but in general we try
to avoid confusion and place it inside the NetBSD MBR partition.

The (single) disklabel can then describe partitions outside of the NetBSD
partition too (as seen when setting up (U)EFI booting on disks with MBR
partitioning - the EFI boot partition shows up in the disklabel).
Post by Rocky Hotas
When the machine is booted with NetBSD, disklabel is the only way NetBSD
looks at the disk, the *entire* disk; MBR in this case is not considered
at all.
That is kind of correct once the kernel is running. For the bootstrap process
(finding and loading the kernel) the MBR is important (both for BIOS and
UEFI boots).
Post by Rocky Hotas
GPT is available to partition a system disk when performing a UEFI
<https://wiki.netbsd.org/Installation_on_UEFI_systems/>
but IIRC a traditional BIOS installation does not offer GPT.
This is not true - you can select GPT for boot disks with both firmwares.
Post by Rocky Hotas
As well, if a new disk is added to an existing NetBSD system (so, it is
not a boot disk), it may be partitioned using only disklabel, not MBR or
GPT.
Actually you have three choices: GPT, MBR+disklabel, disklabel.

You did not stress the (from my POV) most important points of all this mess:

- if you need the disk to be accessible from other systems, you need to
use a partitioning scheme they understand. MBR works for DOS and older
windows, GPT works for nearly everything newer. Disklabel only works
for older Unix systems or NetBSD.
- if your disk is larger than 2TB you can not use MBR (you may have luck
with some systems upto 4TB, but the "portability" is lost), nor can
you use disklabel. Select GPT (sysinst knows about the size limits
and will avoid non-working selections).

The topic is confusing, thanks for summarizing!

Martin

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Rocky Hotas
5 years ago
Permalink
Post by Martin Husemann
I have a few clarifications...
Thanks for writing them.
Post by Martin Husemann
Post by Rocky Hotas
Maybe this is trivial, but during the installation of NetBSD a
disklabel is actually nested inside a primary MBR partition.
This is true for some architectures (like amd64), but others do use plain
disklabel on a disk without any MBR. This also works on amd64 when you
do not need to boot from the disk.
Yes, exactly. I tried to specify this (``Note that the above behaviour
is about amd64; for other ports it may change''), maybe I could have
better positioned the sentence.
...
Oh, ok, maybe I did not dig enough in newer installers.
Post by Martin Husemann
The (single) disklabel can then describe partitions outside of the NetBSD
partition too (as seen when setting up (U)EFI booting on disks with MBR
partitioning - the EFI boot partition shows up in the disklabel).
Thanks for pointing out this.
Post by Martin Husemann
That is kind of correct once the kernel is running.
Yes, I was meaning this.
Post by Martin Husemann
This is not true - you can select GPT for boot disks with both firmwares.
Ops, again my fault about newer installers.
Post by Martin Husemann
- if you need the disk to be accessible from other systems, you need to
use a partitioning scheme they understand. MBR works for DOS and older
windows, GPT works for nearly everything newer. Disklabel only works
for older Unix systems or NetBSD.
I did not specify this about GPT, but as regards MBR DOS compatibility... the
whole message is about this topic. Maybe a recap was missing.
Post by Martin Husemann
- if your disk is larger than 2TB you can not use MBR (you may have luck
with some systems upto 4TB, but the "portability" is lost), nor can
you use disklabel. Select GPT (sysinst knows about the size limits
and will avoid non-working selections).
The topic is confusing, thanks for summarizing!
Thanks to you for all the clarifications and integrations.

Rocky

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Greg Troxel
5 years ago
Permalink
Post by Martin Husemann
- if you need the disk to be accessible from other systems, you need to
use a partitioning scheme they understand. MBR works for DOS and older
windows, GPT works for nearly everything newer. Disklabel only works
for older Unix systems or NetBSD.
Very true, but also: If the disk needs to be readable by some device
which of course has a microcontroller but isn't a "real computer", it's
very likely it can only deal with MBR. Examples are Garmin GPS
receivers, cameras, and things like that.
Rocky Hotas
5 years ago
Permalink
...
Amazing! I never worked with that, so could't guess it. A very good
point!

Rocky

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Bruce Lilly
5 years ago
Permalink
A few additional points:

"disklabel" is a term used by all 3 of the major BSD variants
(FreeBSd, NetBSd, and OpenBSD), but
with significant differences. As a result of those differences, some
of the statements made are not
necessarily accurate in all cases. For example, a type of disklabel
can be written to an OpenBSD
GPT partition rather than to the disk [P]MBR.

Writing a NetBSD disklabel with the eponymous utility will trash the
primary GPT on a GPT-formatted
disk (which can be recovered with NetBSD `gpt recover`).

On GPT disks, only OpenBSD uses an actual physical disklabel structure
on disk; the others refer to
a data structure that is fabricated. NetBSD doesn't appear to
recognize OpenBSD filesystems at
all, and FreeBSD reports OpenBSD's partition as having an "invalid
disklabel". In addition to
disklabel differences, the BSDs use different variations of UFS/FFS,
so filesystems are largely
incompatible.

Incidentally, it would be nice if NetBSD's installer included man
pages or equivalent documentation
(FreeBSD's has man pages and OpenBSD's programs have commands to emit
documentation):
while many systems have programs with similar names and related
functionality, the details are
significantly different, and flying blind is a recipe for disaster.

The systems can (with significant effort) be made to work on a
multiboot system. For example,
here are a few views of one such system. The first is an excerpt of
'dk' lines from NetBSD
dmesg:

[ 3.556584] dk0 at wd0: "EFI system partition", 204800 blocks at
2048, type: msdos
[ 3.583865] dk1 at wd0: "Microsoft reserved partition", 32768
blocks at 206848, type: <unknown>
[ 3.613876] dk2 at wd0: "MS Windows 10", 399767552 blocks at
239616, type: ntfs
[ 3.643883] dk3 at wd0: "FreeBSD swap", 32768000 blocks at
400007168, type: swap
[ 3.673894] dk4 at wd0: "FreeBSD", 399767552 blocks at 432775168, type: ffs
[ 3.703905] dk5 at wd0: "NetBSD", 399773696 blocks at 832542720, type: ffs
[ 3.733915] dk6 at wd0: "NetBSD swap", 32768000 blocks at
1232316416, type: swap
[ 3.763926] dk7 at wd0: "OpenBSD", 399769600 blocks at 1265084416,
type: <unknown>
[ 3.793936] dk8 at wd0: "OpenBSD swap", 32768000 blocks at
1664854016, type: ext2fs
[ 3.823947] dk9 at wd0: "pclinuxos", 399769600 blocks at
1697622016, type: ext2fs
[ 3.853958] dk10 at wd0: "linux-swap", 32768000 blocks at
2097391616, type: <unknown>
[ 3.883968] dk11 at wd0: "data", 1775804416 blocks at 2130159616, type: ntfs
[ 3.913979] dk12 at wd0: "MS recovery", 1062912 blocks at
3905966080, type: <unknown>
[ 5.998829] root on dk5 dumps on dk3

The quoted text is from the GPT partition names, and with the
exception of "OpenBSD swap"
describes their function. Note that NetBSD fails to identify some
filesystem types correctly
(e.g. pclinuxos is XFS, and OpenBSD uses a variant of UFS/FFS). There
also appears to be
a bug where NetBSD claims that it "dumps on" the FreeBSD swap
partition, but there's no
mention of dk3 in /etc/fstab (where dump is supposed to get its information).

The short version of NetBSD's gpt utility provides similar information:

start size index contents
0 1 PMBR
1 1 Pri GPT header
2 32 Pri GPT table
34 2014 Unused
2048 204800 1 GPT part - EFI System
206848 32768 2 GPT part - Windows reserved
239616 399767552 3 GPT part - Windows basic data
400007168 32768000 5 GPT part - FreeBSD swap
432775168 399767552 6 GPT part - FreeBSD UFS/UFS2
832542720 399773696 7 GPT part - NetBSD FFSv1/FFSv2
1232316416 32768000 8 GPT part - NetBSD swap
1265084416 399769600 9 GPT part - 824cc7a0-36a8-11e3-890a-952519ad3f61
1664854016 32768000 10 GPT part - Linux data
1697622016 399769600 11 GPT part - Linux data
2097391616 32768000 12 GPT part - Linux swap
2130159616 1775804416 13 GPT part - Windows basic data
3905964032 2048 Unused
3905966080 1062912 14 GPT part - de94bba4-06d1-4d40-a16a-bfd50179d6ac
3907028992 143 Unused
3907029135 32 Sec GPT table
3907029167 1 Sec GPT header

Note thathe OpenBSD partition is unrecognized, although it has the
exact GUID for
OpenBSD (https://en.wikipedia.org/wiki/GUID_Partition_Table).
Likewise for the MS
Windows recovery environment. The longer version (`gpt show -a`) provides some
additional details. Note that the dk numbering starts at zero,
whereas the gpt indices
are 1-based. The gpt utility is quite useful; I know of only two
shortcomings beyond
the failure to recognize some partition types and filesystems: 1)
short of groping through
embedded strings, there's no version identification for the program,
and 2) there appears
to be no facility for reporting or setting some GPT partition
attributes (hidden, diag, etc.).
`gpt` is the most usable and stable part of the NetBSD installer;
other bits seem to fail
to recognize and/or handle GPT, and much of the documentation is
MBR-specific (w/o
saying so). NetBSD's disklabel command produces output which is
indeed "fictitious":

# /dev/rwd0:
type: ESDI
disk: wd0
label: fictitious
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 16
sectors/cylinder: 1008
cylinders: 3876021
total sectors: 3907029168
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0

4 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 3907029168 0 4.2BSD 0 0 0 # (Cyl. 0
- 3876020)
d: 3907029168 0 unused 0 0 # (Cyl. 0
- 3876020)
disklabel: boot block size 0
disklabel: super block size 0

(for one thing, this particular disk spins at 5400 RPM, not 3600;
NetBSD disklabel
also claims 3600 "rpm" for SSDs and flash drives).

If a GPT label name (up to 36 UTF-16LE characters per Wikipedia)
contains no whitespace
characters, NetBSD's /etc/fstab can use the label, as in

NAME=data /media/dk11 ntfs-3g rw,noatime 0 0

FreeBSD's /etc/fstab allows URI-like hex-encoded escapes for whitespace
characters, so on FreeBSD, the following works:

/dev/gpt/FreeBSD / ufs rw
1 1
/dev/gpt/FreeBSD%20swap none swap sw 0 0
/dev/gpt/data /media/ada0p12 ntfs-3g
rw,noatime,mountprog=/usr/local/bin/ntfs-3g 0 0

FreeBSD creates device files under /dev/gpt (for labels) and /dev/gptid (for
UUIDs) for some partitions (fixed drives, but apparently not removable/portable
media, even when booting from those). FreeBSD has a `gpart` utility.

OpenBSD has its own peculiar fstab notation, so it uses:

0e60da33d75f54ca.b none swap sw
0e60da33d75f54ca.a / ffs rw,wxallowed 1 1
0e60da33d75f54ca.i /boot/EFI msdos rw,noatime 0 0

and mounting an NTFS partition read/write on OpenBSD doesn't work using
fstab (it winds up being read-only, and getting R/W NTFS mounts
requires some trickery to handle automatically at boot time). OpenBSD has its
own variants of `fdisk` and `disklabel` (both with built-in manual pages).

PCLinuxOS fstab is flexible; it can use UUIDs or (non-whitespace) labels, as in:

UUID=5e6c896b-1e1d-437d-8ea9-12590c54c259 / xfs defaults 1 1
UUID=7EEC-CDA9 /boot/EFI vfat defaults 0 0
LABEL=data /media/data ntfs-3g defaults,nofail 0 0

So GPT works reasonably well across various OSes, to the extent that
each OS supports (and documents support for) it; MBR for multiboot of
more than a very few OSes tends to be a problem, and BSD-specific
disklabel is largely an anachronism (and is incompatible between BSD
variants).

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Greg Troxel
5 years ago
Permalink
Post by Bruce Lilly
"disklabel" is a term used by all 3 of the major BSD variants
(FreeBSd, NetBSd, and OpenBSD), but
with significant differences. As a result of those differences, some
of the statements made are not
necessarily accurate in all cases. For example, a type of disklabel
can be written to an OpenBSD
GPT partition rather than to the disk [P]MBR.
Maybe so, but disklabel is a term that originated in 4.3BSD+Tahoe, says
the manpage, and that is not inconsistent with my recollection. So it
should be used to mean that 4.3/4.4-derived structure.

disklabels are usually in reserved space at the beginning of an ffs
filesystem. I have not heard of them being in the MBR.

I can believe someone uses disklabel as a generic term to refer to
various kinds of partioning, but I would suggest that they not do that!
Michael van Elst
5 years ago
Permalink
bruce.lilly+***@gmail.com (Bruce Lilly) writes:

Here is more information :)
Post by Bruce Lilly
There
also appears to be
a bug where NetBSD claims that it "dumps on" the FreeBSD swap
partition, but there's no
mention of dk3 in /etc/fstab (where dump is supposed to get its information).
The kernel choses a default for the dump partition. It might be
necessary before something could read fstab. That default can
be overriden by fstab, since there is only one dump partition
the last entry with type 'dp' is set.
Post by Bruce Lilly
additional details. Note that the dk numbering starts at zero,
whereas the gpt indices
are 1-based.
The dk numbering has nothing to do with the GPT indexes. Would be
difficult with multiple disks anyway. It's just the next free
unit number when a dk instance is created.
Post by Bruce Lilly
to be no facility for reporting or setting some GPT partition
attributes (hidden, diag, etc.).
There are only 3 common attributes defined. The tool has some
additions where needed, but it should probably also allow to
manage all 64 bits.
Post by Bruce Lilly
`gpt` is the most usable and stable part of the NetBSD installer;
The NetBSD installer is called 'sysinst'. You can use the gpt tool when
you exit the installer.
Post by Bruce Lilly
saying so). NetBSD's disklabel command produces output which is
The disklabel command handles the disklabel.
The gpt commands handles GPT.
The fdisk command handles MBR.

The disk drivers maintain an in-core default disklabel which
is named 'fictitious' but which gets updated when a disklabel
can be read from disk. Most people don't change the name though,
so when they write a disklabel, the name persists.

That's one reason why the wedge autodiscover code for disklabel
(not enabled by default) treats the string 'fictitious' as an
empty name.
Post by Bruce Lilly
(for one thing, this particular disk spins at 5400 RPM, not 3600;
NetBSD disklabel
also claims 3600 "rpm" for SSDs and flash drives).
Sure. It's a metric that might be used by a filesystem when allocating
blocks on spinning media. It had little to do with hardware for
quite some time and the filesystem isn't using it either. It's
also from a time where drivers wouldn't reveal such parameters,
instead you used the disklabel to inform the drivers (and filesystems).
Post by Bruce Lilly
If a GPT label name (up to 36 UTF-16LE characters per Wikipedia)
contains no whitespace
characters, NetBSD's /etc/fstab can use the label, as in
NAME=data /media/dk11 ntfs-3g rw,noatime 0 0
The fstab parser understands \ as an escape, so you can specify
white space too.

On the other hand, that's not the GPT label name, but the wedge
name, which is the GPT label name re-encoded as UTF-8 or the GPT
entry UUID if no label is set.

Mounting on /media/dk11 might be misleading, as it might not be
the dk11 device next time.
Post by Bruce Lilly
0e60da33d75f54ca.b none swap sw
0e60da33d75f54ca.a / ffs rw,wxallowed 1 1
0e60da33d75f54ca.i /boot/EFI msdos rw,noatime 0 0
and mounting an NTFS partition read/write on OpenBSD doesn't work using
fstab (it winds up being read-only, and getting R/W NTFS mounts
requires some trickery to handle automatically at boot time).
The NTFS code doesn't support writing.
Post by Bruce Lilly
more than a very few OSes tends to be a problem, and BSD-specific
disklabel is largely an anachronism (and is incompatible between BSD
variants).
The disklabel is surely an anachronism just like MBR is one, as it
cannot represent disks larger than 2TB with 512byte blocks. OpenBSD
has an updated version of the disklabel, but the world seems to
have largely agreed on using GPT.
--
--
Michael van Elst
Internet: ***@serpens.de
"A potential Snark may lurk in every tree."

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Hauke Fath
5 years ago
Permalink
Post by Bruce Lilly
For example, a type of disklabel
can be written to an OpenBSD
GPT partition rather than to the disk [P]MBR.
The NetBSD disklabel(8) will happily do that, too...
Post by Bruce Lilly
Writing a NetBSD disklabel with the eponymous utility will trash the
primary GPT on a GPT-formatted
disk (which can be recovered with NetBSD `gpt recover`).
... unless pointed to the whole disk. I wouldn't expect the OpenBSD
disklabel to act any different here.

Cheerio,
Hauke
--
The ASCII Ribbon Campaign Hauke Fath
() No HTML/RTF in email Institut für Nachrichtentechnik
/\ No Word docs in email TU Darmstadt
Respect for open standards Ruf +49-6151-16-21344

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Martin Husemann
5 years ago
Permalink
Post by Hauke Fath
Post by Bruce Lilly
For example, a type of disklabel
can be written to an OpenBSD
GPT partition rather than to the disk [P]MBR.
The NetBSD disklabel(8) will happily do that, too...
But the kernel won't find it and you can not use the subpartitions, or
am I overlooking something? The whole setup makes no sense at all to me.

Martin

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Hauke Fath
5 years ago
Permalink
Post by Martin Husemann
Post by Hauke Fath
Post by Bruce Lilly
For example, a type of disklabel
can be written to an OpenBSD
GPT partition rather than to the disk [P]MBR.
The NetBSD disklabel(8) will happily do that, too...
But the kernel won't find it and you can not use the subpartitions, or
am I overlooking something? The whole setup makes no sense at all to me.
I guess I was thinking of the workarounds needed on NetBSD before a
gpt-only RAID1 became bootable.

Cheerio,
Hauke
--
The ASCII Ribbon Campaign Hauke Fath
() No HTML/RTF in email Institut für Nachrichtentechnik
/\ No Word docs in email TU Darmstadt
Respect for open standards Ruf +49-6151-16-21344

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Rocky Hotas
5 years ago
Permalink
Post by Bruce Lilly
"disklabel" is a term used by all 3 of the major BSD variants
(FreeBSd, NetBSd, and OpenBSD), but
with significant differences.
A huge "thank you" for the massive amount of information and
observations, for exploring several different cases with pros/cons.

Also, this clarifies several incompatibilities between the *BSD
systems.
Post by Bruce Lilly
`gpt` is the most usable and stable part of the NetBSD installer;
other bits seem to fail
It's a very good tool.
Post by Bruce Lilly
(for one thing, this particular disk spins at 5400 RPM, not 3600;
NetBSD disklabel
also claims 3600 "rpm" for SSDs and flash drives).
Yes, I noticed in my tests that this false value is recurrent.
Thanks also for the comparison of the whitespace tolerance in fstab.
Post by Bruce Lilly
So GPT works reasonably well across various OSes, to the extent that
each OS supports (and documents support for) it; MBR for multiboot of
more than a very few OSes tends to be a problem, and BSD-specific
disklabel is largely an anachronism (and is incompatible between BSD
variants).
This is a very useful recap. As regards the integration of gpt in
NetBSD, I can confirm: it is very well handled by this OS. And your
examples show that gpt is well supported in many other OSs as well.

Yes, maybe disklabel is strongly outdated, but I wanted to share some
notes about it, because I think it was largely used in the past, along
with MBR, with all the issues (and the confusion) we highlighted in
this thread.

Thank you!

Rocky

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Martin Husemann
5 years ago
Permalink
Post by Greg Troxel
disklabels are usually in reserved space at the beginning of an ffs
filesystem. I have not heard of them being in the MBR.
There is an interaction between MBR and disklabel location. If you
have a MBR that specifies a NetBSD partition starting at some sector
N the disklabel will be stored/searched at sector N + LABELSECTOR
(where LABELSECTOR is typicaly 1).

So it is inside the NetBSD MBR partition (not in the MBR itself).

Martin

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Bruce Lilly
5 years ago
Permalink
Post by Martin Husemann
Post by Greg Troxel
disklabels are usually in reserved space at the beginning of an ffs
filesystem. I have not heard of them being in the MBR.
There is an interaction between MBR and disklabel location. If you
have a MBR that specifies a NetBSD partition starting at some sector
N the disklabel will be stored/searched at sector N + LABELSECTOR
(where LABELSECTOR is typicaly 1).
So it is inside the NetBSD MBR partition (not in the MBR itself).
Per the disklabel(8) man page, that might be correct (that "If" is
doing a lot of heavy lifting..):
"On systems that expect to have disks with MBR partitions"
but
"On other systems disklabel will only look at the start of the
disk."

The specific disklabel(8) examples, e.g.

disklabel -i -I sd0

*will* write to the beginning of the disk (i.e. overwriting
at least part of the primary GPT) on a GPT disk.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Martin Husemann
5 years ago
Permalink
Post by Bruce Lilly
Post by Martin Husemann
So it is inside the NetBSD MBR partition (not in the MBR itself).
[..]
Post by Bruce Lilly
The specific disklabel(8) examples, e.g.
disklabel -i -I sd0
*will* write to the beginning of the disk (i.e. overwriting
at least part of the primary GPT) on a GPT disk.
Unless sd0 does have a valid MBR with a NetBSD partition and the
architecture you are trying this on uses MBR/disklabel combined.

There is no point in ever using a disklabel on a disk that has a GPT.

Martin

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Bruce Lilly
5 years ago
Permalink
Post by Martin Husemann
There is no point in ever using a disklabel on a disk that has a GPT.
The various bits of relevant documentation should state so,
clearly and unambiguously.

Instead (for example, not intended to be a complete list of issues), the
current Guide (http://netbsd.org/docs/guide/en/chap-exinst.html#exinst-intro)
has sections for MBR (3.5) and disklabel (3.6), but no mention (at all!)
of GPT (and refers to "BIOS" but has no mention of EFI or UEFI).
3.6 misleadingly states:
"NetBSD uses its own partition scheme, called a disklabel, which is
stored at the start of the MBR partition: "
strongly implying that a physical disklabel is necessary for NetBSD
in all cases, presumably including installation to GPT disks.
It goes on to refer to section 2.2, which is even more outdated
(It refers to "DOS" and has a diagram that looks like a section
of a floppy disc).

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Rocky Hotas
5 years ago
Permalink
Post by Bruce Lilly
Post by Martin Husemann
There is no point in ever using a disklabel on a disk that has a GPT.
This is so important to know. The first time I used GPT I tried in any
way to install a disklabel, too. Then, it became clear it is not
possible.
Post by Bruce Lilly
The various bits of relevant documentation should state so,
clearly and unambiguously.
I quote this.

On one hand, it's not simple to re-write the documentation and maybe
this is the reason why it has not been updated so far (also given the
fact that GPT integration is relatively recent); on the other hand,
this complicates a lot the work of a new user (who doesn't know NetBSD),
which I think should be guided as much as possible.
...
Yes, and this is the reason why I tried to install disklabel on a GPT
partition! Also, the reason why I created this thread.

Rocky

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Bruce Lilly
5 years ago
Permalink
Post by Michael van Elst
The kernel choses a default for the dump partition. It might be
necessary before something could read fstab. That default can
be overriden by fstab, since there is only one dump partition
the last entry with type 'dp' is set.
I noted it because it could be troubling or confusing to some who use
dump. It would be clearer if the dmesg line indicated that the
assignment is "tentative" or "default" or something similar, rather
than the final-sounding "dumps on".
Post by Michael van Elst
Post by Bruce Lilly
to be no facility for reporting or setting some GPT partition
attributes (hidden, diag, etc.).
There are only 3 common attributes defined. The tool has some
additions where needed, but it should probably also allow to
manage all 64 bits.
I'd be happy with at least reporting the flags; gpt appears to do
the right thing e.g. when creating an ESP, but there's no
indication of the boot amd esp flags; worse, `gpt show -a`
reports "Attributes: none", which is misleadingly inaccurate.
Post by Michael van Elst
Post by Bruce Lilly
`gpt` is the most usable and stable part of the NetBSD installer;
The NetBSD installer is called 'sysinst'. You can use the gpt tool when
you exit the installer.
What I was referring to is the whole kit-and-kaboodle that is on the
installation media (CD-ROM or USB).
Post by Michael van Elst
The disklabel command handles the disklabel.
The gpt commands handles GPT.
The fdisk command handles MBR.
[...]
Post by Michael van Elst
Sure. It's a metric that might be used by a filesystem when allocating
blocks on spinning media. It had little to do with hardware for
quite some time and the filesystem isn't using it either. It's
also from a time where drivers wouldn't reveal such parameters,
instead you used the disklabel to inform the drivers (and filesystems).
One big problem is that much of the documentation (man pages,
How-To, NetBSD Guide) is woefully outdated, Half-jokingly,
most of the man pages (etc.) related to MBR and/or disklabel
should include a disclaimer such as:

"Warning! Warning! Danger, Will Robinson!
If you are too young to recognize the context or your hardware
doesn't date from when Bill Mumy was a child actor, you
probably shouldn't use this utility."
Post by Michael van Elst
Post by Bruce Lilly
If a GPT label name (up to 36 UTF-16LE characters per Wikipedia)
contains no whitespace
characters, NetBSD's /etc/fstab can use the label, as in
NAME=data /media/dk11 ntfs-3g rw,noatime 0 0
The fstab parser understands \ as an escape, so you can specify
white space too.
Good to know, but better if it were documented; I see no mention
of that in fstab(5) or the getfsent(3) or getfsspec(3) or
getfsspecname(3) that it references. The examples in fstab(5)
for gpt, viz.

For a gpt(8) disk, use:

NAME=firstpartition / ffs rw,log 1 1
NAME=secondpartition none swap sw,dp 0 0

don't specify whether "firstpartition" etc. are label names or some
magic referring to partition indices, whether or not UUIDs can
be used, and no mention of escapes for whitespace (or raw
or translated UTF...)
Post by Michael van Elst
On the other hand, that's not the GPT label name, but the wedge
name, which is the GPT label name re-encoded as UTF-8 or the GPT
entry UUID if no label is set.
And therein lies a potent source of confusion, as a GPT disk doesn't
have any citrus-y "wedges" (or "slices"), so it's unclear where such a
"wedge name" would come from.

Compare e.g. to fstab on PCLinuxOS, where
UUID=
and
LABEL=
are pretty clear about what's being used.
Post by Michael van Elst
Mounting on /media/dk11 might be misleading, as it might not be
the dk11 device next time.
It's just a directory name (as a mountpoint)...
You are of course correct that the device numbering might change,
which is why the ability to use a UUID or GPT label is
preferable, as those are much less likely to change.
Post by Michael van Elst
Post by Bruce Lilly
and mounting an NTFS partition read/write on OpenBSD doesn't work using
fstab (it winds up being read-only, and getting R/W NTFS mounts
requires some trickery to handle automatically at boot time).
The NTFS code doesn't support writing.
That's true for the plain "ntfs" driver, but not ntfs-3g (or ntfs_3g)
via fuse (or PUFFS). Works fine on all of the systems mentioned,
but (other than MS Windows and Linux) requires a bit of finagling.
For NetBSD in particular, the default kern.sbmax is too small for
the default perfused/puffs/libperfuse buffer (i.e. kernel and PUFFS
subsystem are incompatibly configured as delivered), so either
kern.sbmax has to be increased or PERFUSE_BUFSIZE has
to be decreased.
FreeBSD was a bit easier to get working and OpenBSD a bit
trickier (ntfs-3g can be spec'ed in fstab with some work
(making an appropriate mount_ntfs-3g link), but writing
doesn't work, even though all indications are that the fs is
mouted R/W; the key to getting an automatically-mounted
R/W NTFS partition on OpenBSD is putting a delayed
ntfs-3g command in /etc/rc.local).

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Michael van Elst
5 years ago
Permalink
Post by Bruce Lilly
And therein lies a potent source of confusion, as a GPT disk doesn't
have any citrus-y "wedges" (or "slices"), so it's unclear where such a
"wedge name" would come from.
Just like slices are the partition model used by disklabel, wedges are
the partition model used for other schemes, in particular GPT. So the
disk obviously has wedges and the name somehow comes from GPT.

And you are right, the documentation lacks details.
Post by Bruce Lilly
Compare e.g. to fstab on PCLinuxOS, where
UUID=
and
LABEL=
are pretty clear about what's being used.
That's why you specify NAME= and not LABEL=, i.e. the wedge name (which
can come from other partitioning schemes than GPT).

Also LABEL= and UUID= can be ambigous if you support more than one
partitioning scheme.
...
True, but no fuse filesystem like ntfs-3g is delivered with the
system, and IIRC you get information how to tune the system for it.

Saying that, kern.sbmax is too small for many other unrelated things.
--
--
Michael van Elst
Internet: ***@serpens.de
"A potential Snark may lurk in every tree."

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Rocky Hotas
5 years ago
Permalink
Post by Rocky Hotas
When the machine is booted with NetBSD, disklabel is the only way NetBSD
looks at the disk, the *entire* disk; MBR in this case is not
considered
Post by Rocky Hotas
at all.
Obviously, disklabel creates custom partitions only in the portion of
the disk which is related to NetBSD, the only one where it is intended
to act. It leaves unchanged any other area of the disk. If MBR did not
exist, the extension of disklabel partitions `a', `b', `e', `f', etc.
could span the whole disk, and in fact this is the case of my last
example, where disklabel only is used, without MBR.

In a similar way, when MBR is used from other OSs, it is intended to
create and modify all the primary partitions, except the one where
NetBSD is installed (unless a full disk wipe must be performed): in
other words, it is indended to modify the disk, except the portion
used by NetBSD. This way, NetBSD can coexist with a PC/IBM,
DOS-compatible system.

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Rocky Hotas
5 years ago
Permalink
| Obviously, disklabel creates custom partitions only in the portion of
| the disk which is related to NetBSD,
That's not correct, though if written as a "should create" it would
perhaps be good advice for most people.
In NetBSD, the MBR is used to find the NetBSD partition, and then if
that contains a valid disklabel, that label is used (solely) thereafter.
The label can reference anywhere on the disk, accessing Windows or
[...]
If you're determined to make a mess of the drive, you can create NetBSD
partitions that fit in the middle of a different OS partition - destroying
any data
[...]

Thank you for pointing all this out.
Note that NetBSD is different to FreeBSD here, in FreeBSD each FreeBSD
MBR partition has a disklabel of its own which accesses just the MBR
partition allocated to it.
This was my guess, in fact (``I don't know if FreeBSD also manages in a
different way the coexistence of MBR and disklabel'', without mentioning
the details).

Thanks for all your clarifications!

Rocky

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Greg Troxel
5 years ago
Permalink
Post by Rocky Hotas
Obviously, disklabel creates custom partitions only in the portion of
the disk which is related to NetBSD, the only one where it is intended
to act. It leaves unchanged any other area of the disk. If MBR did not
exist, the extension of disklabel partitions `a', `b', `e', `f', etc.
could span the whole disk, and in fact this is the case of my last
example, where disklabel only is used, without MBR.
This isn't really true. While the installer might only create
partitions within the NetBSD MBR partition, it is entirely feasible and
reasonable to create a disklable partition that refers to some other MBR
partition.y

This is the default on RPI, where you have (skipping uninteresting lines)

Partition table:
0: Primary DOS with 32 bit FAT - LBA (sysid 12)
start 8192, size 188416 (92 MB, Cyls 0/130/3-12/60/48), Active
1: NetBSD (sysid 169)
start 458752, size 61875200 (30213 MB, Cyls 28/141/50-3880/27/51)
PBR is not bootable: All bytes are identical (0x00)

8 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 61875200 458752 4.2BSD 0 0 0 # (Cyl. 224 - 30436*)
b: 262144 196608 swap # (Cyl. 96 - 223)
c: 62333952 0 unused 0 0 # (Cyl. 0 - 30436*)
d: 62333952 0 unused 0 0 # (Cyl. 0 - 30436*)
e: 188416 8192 MSDOS # (Cyl. 4 - 95)

So here 'a' is the NetBSD filesystem within the NetBSD MBR partition,
'b' is the swap partition which is not actually in any MBR partition,
and 'e' is an alias for the MBR partition (used as /boot).

This is how a real RPI install from 8, maybe 7, now 9, has ended up and
that may not be really the best way to do it. But it's an example of
the real complexity of things out there.
Rocky Hotas
5 years ago
Permalink
Post by Greg Troxel
Post by Rocky Hotas
Obviously, disklabel creates custom partitions only in the portion of
the disk which is related to NetBSD, the only one where it is intended
[...]
Post by Greg Troxel
This isn't really true. While the installer might only create
partitions within the NetBSD MBR partition, it is entirely feasible and
reasonable to create a disklable partition that refers to some other MBR
partition.
Yes, this is the same as observed by Robert Elz. As he stated, I should
have written "disklabel should create", "is intended to create", as in
fact I write below, speaking about MBR for the same problem.
Post by Greg Troxel
This is the default on RPI, where you have (skipping uninteresting lines)
[...]

Oh, it's a really concrete example about how diskalbel can act on disk
areas other than the NetBSD MBR primary partition. Yes, maybe it should
be discouraged to avoid confusion, but it has been a necessary
clarification.
Thanks!

Rocky

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-***@muc.de
Loading...