2D pooling operation computed on GPU. More...

#include <pooling2d.h>

Inheritance diagram for Beatmup::NNets::Pooling2D:

Public Types
enum class	Operator { MAX , AVERAGE }
	Pooling operator specification. More...

Public Member Functions
	Pooling2D (const std::string &name, const Operator op, const int size, const int stride=1, const Size::Padding padding=Size::Padding::VALID)
	2D pooling layer. More...

int	getInputCount () const
	Returns number of operation inputs. More...

int	getOutputCount () const
	Returns number of operation outputs. More...

bool	acceptsStorageInput (int index=0) const
	Returns `true` if the operation can take a Storage::View at a specific input. More...

bool	acceptsStorageOutput (int index=0) const
	Returns `true` if the operation can take a Storage::View at a specific output. More...

Size	getOutputSize (int outputIndex=0) const
	Returns full size of a specific operation output. More...

Storage::View	getOutput (int index=0)
	Returns a storage view bound to a specific operation output. More...

void	setInput (Storage::View &&storage, int inputIndex=0)

void	setOutput (Storage::View &&storage, int outputIndex=0)

std::map< std::string, std::string >	serialize () const
	Returns a serialized representation of th operation;. More...

void	disconnect ()
	Assigns empty inputs and outputs. More...

unsigned long	countTexelFetches () const
	Counts (approximate) number of texels fetches. More...

Public Member Functions inherited from Beatmup::NNets::AbstractOperation
virtual	~AbstractOperation ()

virtual bool	usesGpu () const
	Returns `true` if the operation is run on GPU. More...

virtual bool	acceptsVectorInput (int index=0) const
	Returns `true` if the operation can take a GL::Vector at a specific input. More...

virtual bool	acceptsTextureInput (int index=0) const
	Returns `true` if the operation can take a GL::TextureHandler at a specific input. More...

virtual bool	acceptsVectorOutput (int index=0) const
	Returns `true` if the operation can take a GL::Vector at a specific output. More...

virtual bool	acceptsTextureOutput (int index=0) const
	Returns `true` if the operation can take a GL::TextureHandler at a specific output. More...

virtual void	getOutput (GL::Vector *&vector, int index=0)
	Returns a GL::Vector bound to a specific operation output. More...

virtual void	getOutput (GL::TextureHandler *&vector, int index=0)
	Returns a GL::TextureHandler bound to a specific operation output. More...

virtual void	setInput (GL::Vector &vector, int index=0)

virtual void	setOutput (GL::Vector &vector, int index=0)

virtual void	setInput (GL::TextureHandler &image, int index=0)

virtual void	setOutput (GL::TextureHandler &image, int index=0)

virtual unsigned long	countMultiplyAdds () const
	Counts (approximate) number of multiply-adds used by this operation. More...

std::string	getName () const

Static Public Member Functions
static Operator	operatorFromString (const std::string &str)
	Returns a pooling operator from string. More...

static bool	initDeserializer ()
	Sets up deserialization of the operation. More...

Private Member Functions
void	prepare (GraphicPipeline &gpu, ChunkCollection &data, GL::ProgramBank &bank)
	Compiles GLSL shaders. More...

void	execute (TaskThread &thread, GraphicPipeline &gpu)
	Executes the operation. More...

int	getInputPadding (int index=0) const
	Retrieves minimum required size of zero padding for a given input. More...

void	getSampledChannels (int index, int &min, int &max) const
	Retrieves range of input features channels sampled at the same time for a specific input. More...

Private Attributes
const Size	size

const Size	stride

const Operator	op

const Size::Padding	padding

Storage::View	input

Storage::View	output

bool	ready

GL::RenderingProgram *	program

Additional Inherited Members
Protected Member Functions inherited from Beatmup::NNets::AbstractOperation
	AbstractOperation (const std::string &name)

virtual void	execute (TaskThread &thread)
	Executes the operation within a specific CPU thread. More...

Protected Member Functions inherited from Beatmup::NNets::SpatialFilteringMixin
	SpatialFilteringMixin (const int nbSizeX, const int nbSizeY)
	Initializes spatial filtering mixin. More...

	~SpatialFilteringMixin ()

void	writeHeader (StringBuilder &code, bool useUniformShift)
	Writes out the very GLSL fragment shader header required for spatial neighborhood sampling. More...

void	declare (StringBuilder &code, const char *datatype, bool inlineSampling=false)
	Declares GLSL fragment shader main(..) code part required for spatial neighborhood sampling. More...

void	sample (StringBuilder &code, const char inputName, const int inputIndex, const Point &shift, const bool isFirstSample=true, const char suffix="")
	Samples a neighborhood of a given texture. More...

void	sampleInline (StringBuilder &code, const char inputName, const int inputIndex, const IntPoint &position, const Point &shift, const char suffix="")

void	setup (const int width, const int height)
	Prepares the spatial filtering operation execution. More...

void	setUniformShift (GL::Program &program, const IntPoint &shift, const IntPoint &inputSize)
	Applies an offset to the sampling position at runtime. More...

void	setupProgram (GL::Program &program)
	Prepares a given program for spatial filtering. More...

IntRectangle	getSamplingArea (const IntPoint &size, const IntPoint &stride, const Size::Padding padding) const
	Implements common padding policies by computing a rectangular area of positions the sampling kernel takes in order to get the result with the required padding. More...

IntRectangle	getSamplingArea (const Storage::View &storage, const int channel, const IntPoint &stride, const Size::Padding padding) const
	Computes area in pixels to sample a given storage according to specific stride and padding. More...

Rectangle	getTextureCoordinates (const Storage::View &storage, const int channel, const IntPoint &stride, const Size::Padding padding, const IntPoint &outputSize) const
	Computes texture coordinates sampling a specific storage channel for given stride, padding and output size. More...

std::string	getInputSamplingPos () const
	Retrieves input sampling point position for the current fragment. More...

bool	isUniformShiftUsed () const

Static Protected Attributes inherited from Beatmup::NNets::SpatialFilteringMixin
static const char *	SAMPLE_ID_PREFIX = "i"
	prefix of variables declaring a neighbor sample More...

Detailed Description

2D pooling operation computed on GPU.

Has a single input and a single output. Constraints:

Input and output are 3D tensors with values in [0, 1] range sampled over 8 bits.
Number of feature maps is a multiple of 4.
Pooling area is of square shape.
Strides are equal along X and Y.
Average pooling only accepts valid zero padding,

Raspberry Pi-related constraints:

Pi cannot sample more than 256 channels to compute a single output value. Actual practical limit is yet lower: pooling size may be limited by 10. When the limit is reached, Pi OpenGL driver reports an out of memory error (0x505).

Definition at line 40 of file pooling2d.h.

Member Enumeration Documentation

◆ Operator

enum Beatmup::NNets::Pooling2D::Operator

strong

Pooling operator specification.

Enumerator
MAX
AVERAGE

Definition at line 45 of file pooling2d.h.

                                 {
                 MAX,        // max pooling
                 AVERAGE     // average pooling
             };

Constructor & Destructor Documentation

◆ Pooling2D()

Pooling2D::Pooling2D	(	const std::string &	name,
		const Operator	op,
		const int	size,
		const int	stride = `1`,
		const Size::Padding	padding = `Size::Padding::VALID`
	)

2D pooling layer.

Parameters

[in]	name	Layer name
[in]	op	Pooling operator
[in]	size	Spatial pooling operational size
[in]	stride	Pooling stride; if 0, the size is used
[in]	padding	Zero padding applied to the input

Definition at line 29 of file pooling2d.cpp.

  :
     AbstractOperation(name),
     SpatialFilteringMixin(size, size),
     size(size, size, 1), stride(stride <= 0 ? this->size : Size(stride, stride, 1)),
     op(op), padding(padding),
     ready(false), program(nullptr)
 {
     if (op == Operator::AVERAGE)
         InvalidArgument::check(padding == Size::Padding::VALID, "Average 2D pooling only supports valid padding");
     OutOfRange::checkMin(size, 2, "Pooling size must be at least 2, %d got");
 }

Member Function Documentation

◆ prepare()

void Pooling2D::prepare	(	GraphicPipeline &	gpu,
		ChunkCollection &	data,
		GL::ProgramBank &	bank
	)

privatevirtual

Compiles GLSL shaders.

Parameters

[in,out]	gpu	A graphic pipeline instance
[in,out]	data	Chunkfile containing operation data (e.g. weights and biases)
[in,out]	bank	A program bank with existing GLSL programs to be reused when possible. If a new program is built, it is added to the bank.

Implements Beatmup::NNets::AbstractOperation.

Definition at line 48 of file pooling2d.cpp.

                                                                                         {
     // delete the old program
     if (program)
         bank.release(gpu, program);
  
     String code(GL::RenderingPrograms::DECLARE_TEXTURE_COORDINATES_IN_FRAG);
  
     code.printf("uniform sampler2D %s[1];", UNIFORM_INPUT);
     SpatialFilteringMixin::writeHeader(code, true);
  
     // main
     code.line("void main() {");
  
     // declare neighborhood, inline sampling
     SpatialFilteringMixin::declare(code, "lowp vec4", true);
  
     // declare the result variable
     switch (op) {
         case Operator::MAX:
             code("lowp");
         break;
         case Operator::AVERAGE:
             code("mediump");
         break;
     }
     code.line(" vec4 r;");
  
     // compute pooling
     switch (op) {
         case Operator::MAX:
             for (int i = 0; i < size.volume(); ++i) {
                 code(i == 0 ? "r = (" : "r = max(r, ");
                 SpatialFilteringMixin::sampleInline(code, UNIFORM_INPUT, 0, IntPoint(i % size[0], i / size[0]), Point::ZERO);
                 code.line(");");
             }
             code.line("gl_FragColor = r;");
         break;
         case Operator::AVERAGE:
             code("r = ");
             for (int i = 0; i < size.volume(); ++i) {
                 if (i > 0)
                     code(" + ");
                 SpatialFilteringMixin::sampleInline(code, UNIFORM_INPUT, 0, IntPoint(i % size[0], i / size[0]), Point::ZERO);
             }
             code.line(";");
             code.printf("gl_FragColor = %0.10f * r;", 1.0f / size.volume());
         break;
     }
  
     // store the result
     code("}");
  
     // init program
     program = bank(gpu, code);
  
     ready = true;
 }

◆ execute()

void Pooling2D::execute	(	TaskThread &	thread,
		GraphicPipeline &	gpu
	)

privatevirtual

Executes the operation.

The operation should be prepared.

Parameters

[in,out]	thread	Calling CPU thread descriptor
[in,out]	gpu	A graphic pipeline instance

Implements Beatmup::NNets::AbstractOperation.

Definition at line 107 of file pooling2d.cpp.

                                                                 {
     if (!ready)
         throw NotReady(this);
     RuntimeError::check(input, "Input is not provided to a Pooling2D operation.");
     RuntimeError::check(output, "Output is not provided to Pooling2D operation " + getName());
     RuntimeError::check(input.getDepth() == output.getDepth(), "Input / output depth mismatch.");
  
     // prepare deltas
     program->enable(gpu);
     SpatialFilteringMixin::setup(input.getTextureWidth(), input.getTextureHeight());
     SpatialFilteringMixin::setupProgram(*program);
  
     // setup texture coordinates
     gpu.setTextureCoordinates(
         getSamplingArea(input, 0, IntPoint(stride[0], stride[1]), padding),
         input.getTextureSize(),
         output.getSpatialSize()
     );
  
     // for each output channel
     const IntPoint origin = input.getChannelOrigin(0);
     Storage::Binder bind(gpu);
     for (int channel = 0; channel < input.getDepth(); channel += 4) {
         // bind things to program
         bind.begin(*program, output, channel);
         bind(input, UNIFORM_INPUT, channel);
  
         // setup
         SpatialFilteringMixin::setUniformShift(*program, input.getChannelOrigin(channel) - origin, input.getTextureSize());
  
         // run
         program->blend();
     }
 }

◆ getInputPadding()

int Pooling2D::getInputPadding ( int index = 0 ) const

privatevirtual

Retrieves minimum required size of zero padding for a given input.

Operations that sample a neighborhood of a pixel may need the input to be padded with zeros, if some of the neighboring samples fall out of the are containing data. In Beatmup the zero padding is handled by allocating a bigger input and putting zeros around the area that is actually filled with data.

Returns: number of zero columns and rows to be added to the input area.

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 143 of file pooling2d.cpp.

                                               {
     return (padding == Size::Padding::SAME) ? std::max(size[0], size[1]) / 2 : 0;
 }

◆ getSampledChannels()

void Pooling2D::getSampledChannels	(	int	index,
		int &	min,
		int &	max
	)		const

privatevirtual

Retrieves range of input features channels sampled at the same time for a specific input.

The operation would typically take the entire storage and sample it at once, if needed. If the number of textures in a storage exceeds the number of texture samplers that the GPU may use simultaneously, an exception occurs. This function provides the necessary information to limit the number of textures in the storage when allocating it. When the limit is reached, multiple channels are packed into a single texture in the storage.

Parameters

[in]	index	The input index. Expected to fall in the valid range, i.e. from zero to getInputCount() - 1 inclusive.
[out]	min	The minimum number of channels that can be sampled at once
[out]	max	The maximum number of channels that can be sampled at once

Implements Beatmup::NNets::AbstractOperation.

Definition at line 148 of file pooling2d.cpp.

                                                                       {
     min = max = (index == 0 ? 4 : 0);
 }

◆ getInputCount()

int Beatmup::NNets::Pooling2D::getInputCount ( ) const

inlinevirtual

Returns number of operation inputs.

Inputs are then indexed from zero to the returned value minus one inclusive.

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 80 of file pooling2d.h.

80 { return 1; }

◆ getOutputCount()

int Beatmup::NNets::Pooling2D::getOutputCount ( ) const

inlinevirtual

Returns number of operation outputs.

Outputs are then indexed from zero to the returned value minus one inclusive.

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 81 of file pooling2d.h.

81 { return 1; }

◆ acceptsStorageInput()

bool Beatmup::NNets::Pooling2D::acceptsStorageInput ( int index = 0 ) const

inlinevirtual

Returns true if the operation can take a Storage::View at a specific input.

Neural network operations may accept different kinds of data containers on inputs and outputs, namely Storage::View, GL::Vector and textures. This function is used to check whether a given operation accepts a storage view on input.

Parameters

[in] index The input index. Expected to fall in the valid range, i.e. from zero to getInputCount() - 1 inclusive.

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 83 of file pooling2d.h.

83 { return index == 0; }

◆ acceptsStorageOutput()

bool Beatmup::NNets::Pooling2D::acceptsStorageOutput ( int index = 0 ) const

inlinevirtual

Returns true if the operation can take a Storage::View at a specific output.

Neural network operations may accept different kinds of data containers on outputs and outputs, namely Storage::View, GL::Vector and textures. This function is used to check whether a given operation accepts a storage view on output.

Parameters

[in] index The output index. Expected to fall in the valid range, i.e. from zero to getOutputCount() - 1 inclusive.

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 84 of file pooling2d.h.

84 { return index == 0; }

◆ getOutputSize()

Size Pooling2D::getOutputSize ( int outputIndex = 0 ) const

virtual

Returns full size of a specific operation output.

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 153 of file pooling2d.cpp.

                                                    {
     if (outputIndex == 0) {
         RuntimeError::check(input, "Input is not provided to a Pooling2D operation.");
         const Size result = input.getSize().transform(size, stride, padding, input.getDepth());
         RuntimeError::check(result.volume() > 0, "Invalid (zero or negative) output size got in " + getName());
         return result;
     }
     return Size::EMPTY;
 }

◆ getOutput()

Storage::View Beatmup::NNets::Pooling2D::getOutput ( int index = 0 )

inlinevirtual

Returns a storage view bound to a specific operation output.

If no view is bound, returns empty view.

Parameters

[in] index The output index. Expected to fall in the valid range, i.e. from zero to getOutputCount() - 1 inclusive.

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 88 of file pooling2d.h.

88 { return output; }

◆ setInput()

void Pooling2D::setInput	(	Storage::View &&	storage,
		int	inputIndex = `0`
	)

virtual

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 164 of file pooling2d.cpp.

                                                            {
     OutOfRange::check(inputIndex, 0, 0, "Input index out of range: %d");
     RuntimeError::check(view.getStorage().getPadding() >= getInputPadding(inputIndex), "The storage has insufficient padding");
     this->input = std::move(view);
 }

◆ setOutput()

void Pooling2D::setOutput	(	Storage::View &&	storage,
		int	outputIndex = `0`
	)

virtual

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 171 of file pooling2d.cpp.

                                                                 {
     OutOfRange::check(outputIndex, 0, 0, "Output index out of range: %d");
     this->output = std::move(storage);
 }

◆ serialize()

std::map< std::string, std::string > Pooling2D::serialize ( ) const

virtual

Returns a serialized representation of th operation;.

Implements Beatmup::NNets::AbstractOperation.

Definition at line 177 of file pooling2d.cpp.

                                                         {
     return {
         { "_name",      getName() },
         { "_type",      "pooling2d" },
         { "operator",   std::to_string(op) },
         { "size",       std::to_string(size[0]) },
         { "stride",     std::to_string(stride[0]) },
         { "padding",    std::to_string(padding) }
     };
 }

◆ disconnect()

void Pooling2D::disconnect ( )

virtual

Assigns empty inputs and outputs.

Implements Beatmup::NNets::AbstractOperation.

Definition at line 219 of file pooling2d.cpp.

                            {
     this->input = Storage::View();
     this->output = Storage::View();
 }

◆ countTexelFetches()

unsigned long Pooling2D::countTexelFetches ( ) const

virtual

Counts (approximate) number of texels fetches.

Reimplemented from Beatmup::NNets::AbstractOperation.

Definition at line 225 of file pooling2d.cpp.

                                                  {
     return getOutputSize(0).volume() * size.volume() / 4;
 }

◆ operatorFromString()

Pooling2D::Operator Pooling2D::operatorFromString ( const std::string & str )

static

Returns a pooling operator from string.

The conversion is case-insensitive. Raises an exception if cannot interpret the string.

Parameters

[in] str The input string

Definition at line 230 of file pooling2d.cpp.

                                                                     {
     const std::string lc = StringUtils::lowercase(str);
     if (lc == "average")
         return Operator::AVERAGE;
     if (lc == "max")
         return Operator::MAX;
     throw InvalidArgument("Invalid pooling operator: " + str);
     return Operator::AVERAGE;
 }

◆ initDeserializer()

static bool Beatmup::NNets::Pooling2D::initDeserializer ( )

static

Sets up deserialization of the operation.

Member Data Documentation

◆ size

const Size Beatmup::NNets::Pooling2D::size

private

Definition at line 51 of file pooling2d.h.

◆ stride

const Size Beatmup::NNets::Pooling2D::stride

private

Definition at line 51 of file pooling2d.h.

◆ op

const Operator Beatmup::NNets::Pooling2D::op

private

Definition at line 52 of file pooling2d.h.

◆ padding

const Size::Padding Beatmup::NNets::Pooling2D::padding

private

Definition at line 53 of file pooling2d.h.

◆ input

Storage::View Beatmup::NNets::Pooling2D::input

private

Definition at line 54 of file pooling2d.h.

◆ output

Storage::View Beatmup::NNets::Pooling2D::output

private

Definition at line 54 of file pooling2d.h.

◆ ready

bool Beatmup::NNets::Pooling2D::ready

private

Definition at line 55 of file pooling2d.h.

◆ program

GL::RenderingProgram* Beatmup::NNets::Pooling2D::program

private

Definition at line 56 of file pooling2d.h.

The documentation for this class was generated from the following files:

core/nnets/pooling2d.h
core/nnets/pooling2d.cpp

Public Types

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Detailed Description

Member Enumeration Documentation

◆ Operator

Constructor & Destructor Documentation

◆ Pooling2D()

Member Function Documentation

◆ prepare()

◆ execute()

◆ getInputPadding()

◆ getSampledChannels()

◆ getInputCount()

◆ getOutputCount()

◆ acceptsStorageInput()

◆ acceptsStorageOutput()

◆ getOutputSize()

◆ getOutput()

◆ setInput()

◆ setOutput()

◆ serialize()

◆ disconnect()

◆ countTexelFetches()

◆ operatorFromString()

◆ initDeserializer()

Member Data Documentation

◆ size

◆ stride

◆ op

◆ padding

◆ input

◆ output

◆ ready

◆ program