[ipxe-devel] Big endian VS little endian decided during compile time.

[Rusty Weber]

After setting some machines to re-provision themselves through ipxe, I noted that some of the UUID's I gathered from the running OS () of the system did not match the UUID that ipxe returned.
Furthermore, I specifically noted a pattern in which the UUID's returned by ipxe differed in a very big endian to little endian way for the first 8 bytes of the UUID (At first I wasn't really certain why the last half of the ID worked fine). To make matters more complicated, some of the machines in my lab work as expected while others do not.

Example:
(System UUID returned by installed OS) != (System UUID returned by ipxe)
From a Dell R-730
44454c4c-4d00-1051-8037-b8c04f583532 != 4c4c4544-004d-5110-8037-b8c04f583532
From an HP DL-380 gen8
32333536-3030-5355-4532-333343395834 == 32333536-3030-5355-4532-333343395834

The first 8 bytes of the first UUID returned by ipxe in the previous example are wrong, either that or the Linux and windows guys both got their code for reading the value wrong. In any case, those values should match and my investigation started with ipxe.
Investigating where the UUID was being generated from, "./core/uuid.c" I noticed that the first half of the uuid was being printed out and processed by a number of macros defined in "./include/byteswap.h" named "be(16|32)_to_cpu". The names of these macros are as follows:

```
#if __BYTE_ORDER == __LITTLE_ENDIAN
#define __cpu_to_leNN( bits, value ) (value)
#define __cpu_to_beNN( bits, value ) __bswap_ ## bits (value)
#define __leNN_to_cpu( bits, value ) (value)
#define __beNN_to_cpu( bits, value ) __bswap_ ## bits (value)
#define __cpu_to_leNNs( bits, ptr ) do { } while ( 0 )
#define __cpu_to_beNNs( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __leNN_to_cpus( bits, ptr ) do { } while ( 0 )
#define __beNN_to_cpus( bits, ptr ) __bswap_ ## bits ## s (ptr)
#endif

#if __BYTE_ORDER == __BIG_ENDIAN
#define __cpu_to_leNN( bits, value ) __bswap_ ## bits (value)
#define __cpu_to_beNN( bits, value ) (value)
#define __leNN_to_cpu( bits, value ) __bswap_ ## bits (value)
#define __beNN_to_cpu( bits, value ) (value)
#define __cpu_to_leNNs( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __cpu_to_beNNs( bits, ptr ) do { } while ( 0 )
#define __leNN_to_cpus( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __beNN_to_cpus( bits, ptr ) do { } while ( 0 )
#endif

#define be16_to_cpu( value ) __beNN_to_cpu ( 16, value )
#define be32_to_cpu( value ) __beNN_to_cpu ( 32, value )
```

From uuid.c
```
const char * uuid_ntoa ( const union uuid *uuid ) {
static char buf[37]; /* "00000000-0000-0000-0000-000000000000" */

sprintf ( buf, "%08x-%04x-%04x-%04x-%02x%02x%02x%02x%02x%02x",
be32_to_cpu ( uuid->canonical.a ),
be16_to_cpu ( uuid->canonical.b ),
be16_to_cpu ( uuid->canonical.c ),
be16_to_cpu ( uuid->canonical.d ),
uuid->canonical.e[0], uuid->canonical.e[1],
uuid->canonical.e[2], uuid->canonical.e[3],
uuid->canonical.e[4], uuid->canonical.e[5] );
return buf;
```

Here is where I get lost/ am not sure to proceed on, why does the UUID differ from the UUID that the OS returned? Is it possible that there are difference between CPU's where the valued in the UUID are not little endian? Differences in endianness from uefi and the x86 bios images?

Russell Weber
Software Support and Quality engineer

[Christian Nilsson]

Post by Rusty Weber
After setting some machines to re-provision themselves through ipxe, I noted
that some of the UUID’s I gathered from the running OS () of the system did
not match the UUID that ipxe returned.
Furthermore, I specifically noted a pattern in which the UUID’s returned by
ipxe differed in a very big endian to little endian way for the first 8
bytes of the UUID (At first I wasn’t really certain why the last half of the
ID worked fine). To make matters more complicated, some of the machines in
my lab work as expected while others do not.
(System UUID returned by installed OS) != (System UUID returned by ipxe)
From a Dell R-730
44454c4c-4d00-1051-8037-b8c04f583532 != 4c4c4544-004d-5110-8037-b8c04f583532
From an HP DL-380 gen8
32333536-3030-5355-4532-333343395834 == 32333536-3030-5355-4532-333343395834
The first 8 bytes of the first UUID returned by ipxe in the previous example
are wrong, either that or the Linux and windows guys both got their code
for reading the value wrong. In any case, those values should match and my
investigation started with ipxe.
Investigating where the UUID was being generated from, “./core/uuid.c” I
noticed that the first half of the uuid was being printed out and processed
by a number of macros defined in “./include/byteswap.h“ named
```
#if __BYTE_ORDER == __LITTLE_ENDIAN
#define __cpu_to_leNN( bits, value ) (value)
#define __cpu_to_beNN( bits, value ) __bswap_ ## bits (value)
#define __leNN_to_cpu( bits, value ) (value)
#define __beNN_to_cpu( bits, value ) __bswap_ ## bits (value)
#define __cpu_to_leNNs( bits, ptr ) do { } while ( 0 )
#define __cpu_to_beNNs( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __leNN_to_cpus( bits, ptr ) do { } while ( 0 )
#define __beNN_to_cpus( bits, ptr ) __bswap_ ## bits ## s (ptr)
#endif
#if __BYTE_ORDER == __BIG_ENDIAN
#define __cpu_to_leNN( bits, value ) __bswap_ ## bits (value)
#define __cpu_to_beNN( bits, value ) (value)
#define __leNN_to_cpu( bits, value ) __bswap_ ## bits (value)
#define __beNN_to_cpu( bits, value ) (value)
#define __cpu_to_leNNs( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __cpu_to_beNNs( bits, ptr ) do { } while ( 0 )
#define __leNN_to_cpus( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __beNN_to_cpus( bits, ptr ) do { } while ( 0 )
#endif
#define be16_to_cpu( value ) __beNN_to_cpu ( 16, value )
#define be32_to_cpu( value ) __beNN_to_cpu ( 32, value )
```
From uuid.c
```
const char * uuid_ntoa ( const union uuid *uuid ) {
static char buf[37]; /* "00000000-0000-0000-0000-000000000000" */
sprintf ( buf, "%08x-%04x-%04x-%04x-%02x%02x%02x%02x%02x%02x",
be32_to_cpu ( uuid->canonical.a ),
be16_to_cpu ( uuid->canonical.b ),
be16_to_cpu ( uuid->canonical.c ),
be16_to_cpu ( uuid->canonical.d ),
uuid->canonical.e[0], uuid->canonical.e[1],
uuid->canonical.e[2], uuid->canonical.e[3],
uuid->canonical.e[4], uuid->canonical.e[5] );
return buf;
```
Here is where I get lost/ am not sure to proceed on, why does the UUID
differ from the UUID that the OS returned? Is it possible that there are
difference between CPU’s where the valued in the UUID are not little endian?
Differences in endianness from uefi and the x86 bios images?
Russell Weber
Software Support and Quality engineer

Why this is as it is might be explained by this commit:
https://git.ipxe.org/ipxe.git/commitdiff/9e896d0eeaa07d47b2bed4c92072fd638ce3eb55

[smbios] Mangle UUIDs for SMBIOS version 2.6 and newer

iPXE treats UUIDs as being in network byte order (big-endian). The
SMBIOS specification version 2.6 states that UUIDs are stored with
little-endian values in the first three fields; earlier versions did
not specify an endianness. This results in some inconsistency between
the BIOS, vendor PXE, iPXE, and operating system interpretations of
the SMBIOS UUID.

dmidecode assumes that the byte order is little-endian if and only if
the SMBIOS version is 2.6 or higher. Choose to match this behaviour.

[Rusty Weber]

That would make sense if the smbios version that I have wasn't greater than 2.6.

At least, that is what is reported by the Host os:

PNP BIOS 1.0 present.
Event Notification: Not Supported
Real Mode 16-bit Code Address: F000:E2F1
Real Mode 16-bit Data Address: 0040:0000
16-bit Protected Mode Code Address: 0x000FE2F4
16-bit Protected Mode Data Address: 0x00000040
SMBIOS 2.8 present.
Structure Table Length: 4109 bytes
Structure Table Address: 0x7A028000
Number Of Structures: 79
Maximum Structure Size: 468 bytes


However the version that shows in ipxe is:
echo ${smbios/0.5.0}
1.3.6

Hmm...

-----Original Message-----
From: Christian Nilsson [mailto:***@gmail.com]
Sent: Friday, April 20, 2018 12:41 AM
To: Rusty Weber
Cc: ipxe-***@lists.ipxe.org
Subject: Re: [ipxe-devel] Big endian VS little endian decided during compile time.

Post by Rusty Weber
After setting some machines to re-provision themselves through ipxe, I
noted that some of the UUID’s I gathered from the running OS () of the
system did not match the UUID that ipxe returned.
Furthermore, I specifically noted a pattern in which the UUID’s
returned by ipxe differed in a very big endian to little endian way
for the first 8 bytes of the UUID (At first I wasn’t really certain
why the last half of the ID worked fine). To make matters more
complicated, some of the machines in my lab work as expected while others do not.
(System UUID returned by installed OS) != (System UUID returned by
ipxe) From a Dell R-730
44454c4c-4d00-1051-8037-b8c04f583532 !=
4c4c4544-004d-5110-8037-b8c04f583532
From an HP DL-380 gen8
32333536-3030-5355-4532-333343395834 ==
32333536-3030-5355-4532-333343395834
The first 8 bytes of the first UUID returned by ipxe in the previous
example are wrong, either that or the Linux and windows guys both got
their code for reading the value wrong. In any case, those values
should match and my investigation started with ipxe.
Investigating where the UUID was being generated from, “./core/uuid.c”
I noticed that the first half of the uuid was being printed out and
processed by a number of macros defined in “./include/byteswap.h“
```
#if __BYTE_ORDER == __LITTLE_ENDIAN
#define __cpu_to_leNN( bits, value ) (value)
#define __cpu_to_beNN( bits, value ) __bswap_ ## bits (value)
#define __leNN_to_cpu( bits, value ) (value)
#define __beNN_to_cpu( bits, value ) __bswap_ ## bits (value)
#define __cpu_to_leNNs( bits, ptr ) do { } while ( 0 )
#define __cpu_to_beNNs( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __leNN_to_cpus( bits, ptr ) do { } while ( 0 )
#define __beNN_to_cpus( bits, ptr ) __bswap_ ## bits ## s (ptr)
#endif
#if __BYTE_ORDER == __BIG_ENDIAN
#define __cpu_to_leNN( bits, value ) __bswap_ ## bits (value)
#define __cpu_to_beNN( bits, value ) (value)
#define __leNN_to_cpu( bits, value ) __bswap_ ## bits (value)
#define __beNN_to_cpu( bits, value ) (value)
#define __cpu_to_leNNs( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __cpu_to_beNNs( bits, ptr ) do { } while ( 0 )
#define __leNN_to_cpus( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __beNN_to_cpus( bits, ptr ) do { } while ( 0 )
#endif
#define be16_to_cpu( value ) __beNN_to_cpu ( 16, value )
#define be32_to_cpu( value ) __beNN_to_cpu ( 32, value )
```
From uuid.c
```
const char * uuid_ntoa ( const union uuid *uuid ) {
static char buf[37]; /* "00000000-0000-0000-0000-000000000000" */
sprintf ( buf, "%08x-%04x-%04x-%04x-%02x%02x%02x%02x%02x%02x",
be32_to_cpu ( uuid->canonical.a ),
be16_to_cpu ( uuid->canonical.b ),
be16_to_cpu ( uuid->canonical.c ),
be16_to_cpu ( uuid->canonical.d ),
uuid->canonical.e[0], uuid->canonical.e[1],
uuid->canonical.e[2], uuid->canonical.e[3],
uuid->canonical.e[4], uuid->canonical.e[5] );
return buf;
```
Here is where I get lost/ am not sure to proceed on, why does the UUID
differ from the UUID that the OS returned? Is it possible that there
are difference between CPU’s where the valued in the UUID are not little endian?
Differences in endianness from uefi and the x86 bios images?
Russell Weber
Software Support and Quality engineer

Why this is as it is might be explained by this commit:
https://git.ipxe.org/ipxe.git/commitdiff/9e896d0eeaa07d47b2bed4c92072fd638ce3eb55

[smbios] Mangle UUIDs for SMBIOS version 2.6 and newer

iPXE treats UUIDs as being in network byte order (big-endian). The SMBIOS specification version 2.6 states that UUIDs are stored with little-endian values in the first three fields; earlier versions did not specify an endianness. This results in some inconsistency between the BIOS, vendor PXE, iPXE, and operating system interpretations of the SMBIOS UUID.

dmidecode assumes that the byte order is little-endian if and only if the SMBIOS version is 2.6 or higher. Choose to match this behaviour.

[Rusty Weber]

Doing a bit more research..
echo ${smbios/0.5.0}
1.3.6
Gives me the actual bios version, not the smbios version.
Attempting to update the bios gives me the same results in uuid.
echo ${smbios/0.5.0}
2.7.1
-----Original Message-----
From: Rusty Weber
Sent: Friday, April 20, 2018 4:22 AM
To: 'Christian Nilsson'
Cc: ipxe-***@lists.ipxe.org
Subject: RE: [ipxe-devel] Big endian VS little endian decided during compile time.

That would make sense if the smbios version that I have wasn't greater than 2.6.

At least, that is what is reported by the Host os:

PNP BIOS 1.0 present.
Event Notification: Not Supported
Real Mode 16-bit Code Address: F000:E2F1
Real Mode 16-bit Data Address: 0040:0000
16-bit Protected Mode Code Address: 0x000FE2F4
16-bit Protected Mode Data Address: 0x00000040 SMBIOS 2.8 present.
Structure Table Length: 4109 bytes
Structure Table Address: 0x7A028000
Number Of Structures: 79
Maximum Structure Size: 468 bytes


However the version that shows in ipxe is:
echo ${smbios/0.5.0}
1.3.6

Hmm...

-----Original Message-----
From: Christian Nilsson [mailto:***@gmail.com]
Sent: Friday, April 20, 2018 12:41 AM
To: Rusty Weber
Cc: ipxe-***@lists.ipxe.org
Subject: Re: [ipxe-devel] Big endian VS little endian decided during compile time.

Post by Rusty Weber
After setting some machines to re-provision themselves through ipxe, I
noted that some of the UUID’s I gathered from the running OS () of the
system did not match the UUID that ipxe returned.
Furthermore, I specifically noted a pattern in which the UUID’s
returned by ipxe differed in a very big endian to little endian way
for the first 8 bytes of the UUID (At first I wasn’t really certain
why the last half of the ID worked fine). To make matters more
complicated, some of the machines in my lab work as expected while others do not.
(System UUID returned by installed OS) != (System UUID returned by
ipxe) From a Dell R-730
44454c4c-4d00-1051-8037-b8c04f583532 !=
4c4c4544-004d-5110-8037-b8c04f583532
From an HP DL-380 gen8
32333536-3030-5355-4532-333343395834 ==
32333536-3030-5355-4532-333343395834
The first 8 bytes of the first UUID returned by ipxe in the previous
example are wrong, either that or the Linux and windows guys both got
their code for reading the value wrong. In any case, those values
should match and my investigation started with ipxe.
Investigating where the UUID was being generated from, “./core/uuid.c”
I noticed that the first half of the uuid was being printed out and
processed by a number of macros defined in “./include/byteswap.h“
```
#if __BYTE_ORDER == __LITTLE_ENDIAN
#define __cpu_to_leNN( bits, value ) (value)
#define __cpu_to_beNN( bits, value ) __bswap_ ## bits (value)
#define __leNN_to_cpu( bits, value ) (value)
#define __beNN_to_cpu( bits, value ) __bswap_ ## bits (value)
#define __cpu_to_leNNs( bits, ptr ) do { } while ( 0 )
#define __cpu_to_beNNs( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __leNN_to_cpus( bits, ptr ) do { } while ( 0 )
#define __beNN_to_cpus( bits, ptr ) __bswap_ ## bits ## s (ptr)
#endif
#if __BYTE_ORDER == __BIG_ENDIAN
#define __cpu_to_leNN( bits, value ) __bswap_ ## bits (value)
#define __cpu_to_beNN( bits, value ) (value)
#define __leNN_to_cpu( bits, value ) __bswap_ ## bits (value)
#define __beNN_to_cpu( bits, value ) (value)
#define __cpu_to_leNNs( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __cpu_to_beNNs( bits, ptr ) do { } while ( 0 )
#define __leNN_to_cpus( bits, ptr ) __bswap_ ## bits ## s (ptr)
#define __beNN_to_cpus( bits, ptr ) do { } while ( 0 )
#endif
#define be16_to_cpu( value ) __beNN_to_cpu ( 16, value )
#define be32_to_cpu( value ) __beNN_to_cpu ( 32, value )
```
From uuid.c
```
const char * uuid_ntoa ( const union uuid *uuid ) {
static char buf[37]; /* "00000000-0000-0000-0000-000000000000" */
sprintf ( buf, "%08x-%04x-%04x-%04x-%02x%02x%02x%02x%02x%02x",
be32_to_cpu ( uuid->canonical.a ),
be16_to_cpu ( uuid->canonical.b ),
be16_to_cpu ( uuid->canonical.c ),
be16_to_cpu ( uuid->canonical.d ),
uuid->canonical.e[0], uuid->canonical.e[1],
uuid->canonical.e[2], uuid->canonical.e[3],
uuid->canonical.e[4], uuid->canonical.e[5] );
return buf;
```
Here is where I get lost/ am not sure to proceed on, why does the UUID
differ from the UUID that the OS returned? Is it possible that there
are difference between CPU’s where the valued in the UUID are not little endian?
Differences in endianness from uefi and the x86 bios images?
Russell Weber
Software Support and Quality engineer

Why this is as it is might be explained by this commit:
https://git.ipxe.org/ipxe.git/commitdiff/9e896d0eeaa07d47b2bed4c92072fd638ce3eb55

[smbios] Mangle UUIDs for SMBIOS version 2.6 and newer

iPXE treats UUIDs as being in network byte order (big-endian). The SMBIOS specification version 2.6 states that UUIDs are stored with little-endian values in the first three fields; earlier versions did not specify an endianness. This results in some inconsistency between the BIOS, vendor PXE, iPXE, and operating system interpretations of the SMBIOS UUID.

dmidecode assumes that the byte order is little-endian if and only if the SMBIOS version is 2.6 or higher. Choose to match this behaviour.