SIMD - Single Instruction Multiple Data

Photo by Vladimir Patkachakov on Unsplash

Hi! Today’s lecture, we learned SIMD - Single Instruction Multiple Data. This is a great tool to process data in a bulk fashion. So, instead of doing one by one, based on the variable size, we can do 16, 8, 4 or 2 at the time. This technique is called auto-vectorization resources, and it falls into the category of machine instruction optimization that I mentioned in my last post.

If the machine is SIMD enabled, the compiler can use it when translating a sum loop, for example. If we are summing 8 bits numbers, using SIMD, it will be 16 times faster. However, the compiler can figure that it is not safe to use SIMD due to overlapping or non-aligned data. In fact, the compiler will not apply SIMD in most cases, so we need to get our hands dirty and inject some assembly. I’ll show you how to do it in a second.

Here are the lanes of the 128-bit AArch64 Advanced SIMD:
16 x 8 bits
8 x 16 bits
4 x 32 bits
2 x 64 bits
1 x 128 bits

Reading the ARM manual, we can find a lot of SIMD functions. Bringing back the volume example, we can process 8 values each time, and not worry about the overflow. The magic instruction is SQDMULH – Signed Integer Saturating Doubling Multiply returning High Half. With that name, it must make coffee too! Well, no. It multiplies the first parameter with the second. It puts the result into the third, discarding the fraction portion and not overflowing – on overflow, it will keep the minimum or maximum value. It is precisely what we need to deal with the volume in one instruction.

Now let’s mix some C and Assembly, shall we?

The syntax is:
__asm__ ("assembly code" : outputs : inputs : clobbers);

Warning: this will break the portability. It is a good idea to have compiler flags to “pick” the right portion of the code based on the architecture being compiled. Here we are not doing that.

This is the code provided by our instructor. Do you see the loop in C and the ASM instruction inside? The line 52 is doing 8 values per iteration using the magic single instruction SQDMULH. It is fast! The code, as it is, will only work on Arch64, though.

	// vol_inline.c :: volume scaling in C using AArch64 SIMD
	// Chris Tyler 2017.11.29-2019.10.02 - Licensed under GPLv3.
	// For the SIMD lab in the Seneca College SPO600 Course
	#include <stdlib.h>
	#include <stdio.h>
	#include <stdint.h>
	#include "vol.h"
	int main() {
	int16_t* data; // input array
	int16_t* limit; // end of input array
	// these variables will be used in our assembler code, so we're going
	// to hand-allocate which register they are placed in
	// Q: what is an alternate approach?
	register int16_t* cursor asm("r20"); // input cursor
	register int16_t vol_int asm("r22"); // volume as int16_t
	int x; // array interator
	int ttl =0 ; // array total
	data=(int16_t*) calloc(SAMPLES, sizeof(int16_t));
	srand(-1);
	printf("Generating sample data.\n");
	for (x = 0; x < SAMPLES; x++) {
	data[x] = (rand()%65536)-32768;
	}
	// --------------------------------------------------------------------
	cursor = data;
	limit = data+ SAMPLES ;
	// set vol_int to fixed-point representation of 0.75
	// Q: should we use 32767 or 32768 in next line? why?
	vol_int = (int16_t) (0.75 * 32767.0);
	printf("Scaling samples.\n");
	// Q: what does it mean to "duplicate" values in the next line?
	__asm__ ("dup v1.8h,%w0"::"r"(vol_int)); // duplicate vol_int into v1.8h
	while ( cursor < limit ) {
	__asm__ (
	"ldr q0, [%[cursor]], #0 \n\t"
	// load eight samples into q0 (v0.8h)
	// from in_cursor
	"sqdmulh v0.8h, v0.8h, v1.8h \n\t"
	// multiply each lane in v0 by v1*2
	// saturate results
	// store upper 16 bits of results into
	// the corresponding lane in v0
	// Q: Why is #16 included in the str line
	// but not in the ldr line?
	"str q0, [%[cursor]],#16 \n\t"
	// store eight samples to [cursor]
	// post-increment cursor by 16 bytes
	// and store back into the pointer register
	// Q: What do these next three lines do?
	: [cursor]"+r"(cursor)
	: "r"(cursor)
	: "memory"
	);
	}
	// --------------------------------------------------------------------
	printf("Summing samples.\n");
	for (x = 0; x < SAMPLES; x++) {
	ttl=(ttl+data[x])%1000;
	}
	// Q: are the results usable? are they correct?
	printf("Result: %d\n", ttl);
	return 0;
	}

view raw vol_inline.c hosted with ❤ by GitHub

If you don’t like assembly like me, intrinsics will help. The GCC compiler has some sort of functions representations of the assembly instructions. I think that it helps, but it also has its limitations. Here is the same example, but using intrinsics. Take a look at line 42.

	// vol_intrinsics.c :: volume scaling in C using AArch64 Intrinsics
	// Chris Tyler 2019.10.02 - Licensed under GPLv3
	// For the SIMD lab in the Seneca College SPO600 Course
	#include <stdlib.h>
	#include <stdio.h>
	#include <stdint.h>
	#include <arm_neon.h>
	#include "vol.h"
	int main() {
	int16_t* data; // data array
	int16_t* limit; // end of input array
	register int16_t* cursor asm("r20"); // array cursor (pointer)
	register int16_t vol_int asm("r22"); // volume as int16_t
	int x; // array interator
	int ttl = 0; // array total
	data=(int16_t*) calloc(SAMPLES, sizeof(int16_t));
	srand(-1);
	printf("Generating sample data.\n");
	for (x = 0; x < SAMPLES; x++) {
	data[x] = (rand()%65536)-32768;
	}
	// --------------------------------------------------------------------
	cursor = data; // Pointer to start of array
	limit = data + SAMPLES ;
	vol_int = (int16_t) (0.75 * 32767.0);
	printf("Scaling samples.\n");
	while ( cursor < limit ) {
	// Q: What do these intrinsic functions do?
	// (See gcc intrinsics documentation)
	vst1q_s16(cursor, vqdmulhq_s16(vld1q_s16(cursor), vdupq_n_s16(vol_int)));
	// Q: Why is the increment below 8 instead of 16 or some other value?
	// Q: Why is this line not needed in the inline assembler version
	// of this program?
	cursor += 8;
	}
	// --------------------------------------------------------------------
	printf("Summing samples.\n");
	for (x = 0; x < SAMPLES; x++) {
	ttl=(ttl+data[x])%1000;
	}
	// Q: Are the results usable? Are they accurate?
	printf("Result: %d\n", ttl);
	return 0;
	}

view raw vol_intrinsics.c hosted with ❤ by GitHub

This is it for today. I’m working on profiling my awk build. Stay tuned!