Abstract neural net operation (layer). More...

#include <operation.h>

Inheritance diagram for Beatmup::NNets::AbstractOperation:

Classes
class	Deserializer
	Enables construction of an operation from its serialized representation. More...

class	InconsistentModelData

class	NotReady

Public Member Functions
virtual	~AbstractOperation ()

virtual bool	usesGpu () const
	Returns `true` if the operation is run on GPU. More...

virtual int	getInputCount () const
	Returns number of operation inputs. More...

virtual int	getOutputCount () const
	Returns number of operation outputs. More...

virtual bool	acceptsStorageInput (int index=0) const
	Returns `true` if the operation can take a Storage::View at a specific input. More...

virtual bool	acceptsVectorInput (int index=0) const
	Returns `true` if the operation can take a GL::Vector at a specific input. More...

virtual bool	acceptsTextureInput (int index=0) const
	Returns `true` if the operation can take a GL::TextureHandler at a specific input. More...

virtual bool	acceptsStorageOutput (int index=0) const
	Returns `true` if the operation can take a Storage::View at a specific output. More...

virtual bool	acceptsVectorOutput (int index=0) const
	Returns `true` if the operation can take a GL::Vector at a specific output. More...

virtual bool	acceptsTextureOutput (int index=0) const
	Returns `true` if the operation can take a GL::TextureHandler at a specific output. More...

virtual Size	getOutputSize (int outputIndex=0) const
	Returns full size of a specific operation output. More...

virtual Storage::View	getOutput (int index=0)
	Returns a storage view bound to a specific operation output. More...

virtual void	getOutput (GL::Vector *&vector, int index=0)
	Returns a GL::Vector bound to a specific operation output. More...

virtual void	getOutput (GL::TextureHandler *&vector, int index=0)
	Returns a GL::TextureHandler bound to a specific operation output. More...

virtual void	setInput (Storage::View &&storage, int index=0)

virtual void	setOutput (Storage::View &&storage, int index=0)

virtual void	setInput (GL::Vector &vector, int index=0)

virtual void	setOutput (GL::Vector &vector, int index=0)

virtual void	setInput (GL::TextureHandler &image, int index=0)

virtual void	setOutput (GL::TextureHandler &image, int index=0)

virtual std::map< std::string, std::string >	serialize () const =0
	Returns a serialized representation of th operation;. More...

virtual void	disconnect ()=0
	Assigns empty inputs and outputs. More...

virtual unsigned long	countMultiplyAdds () const
	Counts (approximate) number of multiply-adds used by this operation. More...

virtual unsigned long	countTexelFetches () const
	Counts (approximate) number of texels fetches. More...

std::string	getName () const

Protected Member Functions
	AbstractOperation (const std::string &name)

virtual void	prepare (GraphicPipeline &gpu, ChunkCollection &data, GL::ProgramBank &bank)=0
	Compiles GLSL shaders. More...

virtual void	execute (TaskThread &thread, GraphicPipeline &gpu)=0
	Executes the operation. More...

virtual void	execute (TaskThread &thread)
	Executes the operation within a specific CPU thread. More...

virtual int	getInputPadding (int index=0) const
	Retrieves minimum required size of zero padding for a given input. More...

virtual void	getSampledChannels (int index, int &min, int &max) const =0
	Retrieves range of input features channels sampled at the same time for a specific input. More...

Private Member Functions
	AbstractOperation (const AbstractOperation &)=delete
	disabling copying constructor More...

Private Attributes
std::string	name

Friends
class	Model

Detailed Description

Abstract neural net operation (layer).

Has a name used to refer the operation in a Model. The operation data (such as convolution weights) is provided through a ChunkCollection in single precision floating point format, where the chunks are searched by operation name. Operations have several inputs and outputs numbered starting from zero. Different operations accept different kinds of input and output data.

Inputs may be Storage, GL::TextureHandler or GL::Vector.
Outputs may be Storage or GL::Vector. The operations are built in a deferred fashion during the first inference run, which makes the first run much slower than the subsequent runs.

Definition at line 46 of file operation.h.

Constructor & Destructor Documentation

◆ AbstractOperation() [1/2]

Beatmup::NNets::AbstractOperation::AbstractOperation ( const AbstractOperation & )

privatedelete

disabling copying constructor

◆ AbstractOperation() [2/2]

Beatmup::NNets::AbstractOperation::AbstractOperation ( const std::string & name )

inlineprotected

Definition at line 53 of file operation.h.

53 : name(name) {}

Beatmup::NNets::AbstractOperation::name

std::string name

Definition: operation.h:50

◆ ~AbstractOperation()

virtual Beatmup::NNets::AbstractOperation::~AbstractOperation ( )

inlinevirtual

Definition at line 117 of file operation.h.

117 {}

Member Function Documentation

◆ prepare()

virtual void Beatmup::NNets::AbstractOperation::prepare	(	GraphicPipeline &	gpu,
		ChunkCollection &	data,
		GL::ProgramBank &	bank
	)

protectedpure virtual

Compiles GLSL shaders.

Parameters

[in,out]	gpu	A graphic pipeline instance
[in,out]	data	Chunkfile containing operation data (e.g. weights and biases)
[in,out]	bank	A program bank with existing GLSL programs to be reused when possible. If a new program is built, it is added to the bank.

Implemented in Beatmup::NNets::Pooling2D, Beatmup::NNets::CpuOperation, Beatmup::NNets::ImageSampler, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

◆ execute() [1/2]

virtual void Beatmup::NNets::AbstractOperation::execute	(	TaskThread &	thread,
		GraphicPipeline &	gpu
	)

protectedpure virtual

Executes the operation.

The operation should be prepared.

Parameters

[in,out]	thread	Calling CPU thread descriptor
[in,out]	gpu	A graphic pipeline instance

Implemented in Beatmup::NNets::Pooling2D, Beatmup::NNets::CpuOperation, Beatmup::NNets::ImageSampler, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

◆ execute() [2/2]

virtual void Beatmup::NNets::AbstractOperation::execute ( TaskThread & thread )

inlineprotectedvirtual

Executes the operation within a specific CPU thread.

Parameters

[in,out] thread Calling CPU thread descriptor

Reimplemented in Beatmup::NNets::CpuOperation.

Definition at line 76 of file operation.h.

76 {}

◆ getInputPadding()

virtual int Beatmup::NNets::AbstractOperation::getInputPadding ( int index = 0 ) const

inlineprotectedvirtual

Retrieves minimum required size of zero padding for a given input.

Operations that sample a neighborhood of a pixel may need the input to be padded with zeros, if some of the neighboring samples fall out of the are containing data. In Beatmup the zero padding is handled by allocating a bigger input and putting zeros around the area that is actually filled with data.

Returns: number of zero columns and rows to be added to the input area.

Reimplemented in Beatmup::NNets::Pooling2D, Beatmup::NNets::ImageSampler, and Beatmup::NNets::Conv2D.

Definition at line 85 of file operation.h.

85 { return 0; }

◆ getSampledChannels()

virtual void Beatmup::NNets::AbstractOperation::getSampledChannels	(	int	index,
		int &	min,
		int &	max
	)		const

protectedpure virtual

Retrieves range of input features channels sampled at the same time for a specific input.

The operation would typically take the entire storage and sample it at once, if needed. If the number of textures in a storage exceeds the number of texture samplers that the GPU may use simultaneously, an exception occurs. This function provides the necessary information to limit the number of textures in the storage when allocating it. When the limit is reached, multiple channels are packed into a single texture in the storage.

Parameters

[in]	index	The input index. Expected to fall in the valid range, i.e. from zero to getInputCount() - 1 inclusive.
[out]	min	The minimum number of channels that can be sampled at once
[out]	max	The maximum number of channels that can be sampled at once

Implemented in Beatmup::NNets::Pooling2D, Beatmup::NNets::CpuOperation, Beatmup::NNets::ImageSampler, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

◆ usesGpu()

virtual bool Beatmup::NNets::AbstractOperation::usesGpu ( ) const

inlinevirtual

Returns true if the operation is run on GPU.

Otherwise, it is run on CPU.

Reimplemented in Beatmup::NNets::CpuOperation.

Definition at line 123 of file operation.h.

123 { return true; }

◆ getInputCount()

virtual int Beatmup::NNets::AbstractOperation::getInputCount ( ) const

inlinevirtual

Returns number of operation inputs.

Inputs are then indexed from zero to the returned value minus one inclusive.

Reimplemented in Beatmup::NNets::Softmax, Beatmup::NNets::Pooling2D, Beatmup::NNets::ImageSampler, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

Definition at line 129 of file operation.h.

129 { return 0; }

◆ getOutputCount()

virtual int Beatmup::NNets::AbstractOperation::getOutputCount ( ) const

inlinevirtual

Returns number of operation outputs.

Outputs are then indexed from zero to the returned value minus one inclusive.

Reimplemented in Beatmup::NNets::Softmax, Beatmup::NNets::Pooling2D, Beatmup::NNets::ImageSampler, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

Definition at line 135 of file operation.h.

135 { return 0; }

◆ acceptsStorageInput()

virtual bool Beatmup::NNets::AbstractOperation::acceptsStorageInput ( int index = 0 ) const

inlinevirtual

Returns true if the operation can take a Storage::View at a specific input.

Neural network operations may accept different kinds of data containers on inputs and outputs, namely Storage::View, GL::Vector and textures. This function is used to check whether a given operation accepts a storage view on input.

Parameters

[in] index The input index. Expected to fall in the valid range, i.e. from zero to getInputCount() - 1 inclusive.

Reimplemented in Beatmup::NNets::Softmax, Beatmup::NNets::Pooling2D, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

Definition at line 143 of file operation.h.

143 { return false; }

◆ acceptsVectorInput()

virtual bool Beatmup::NNets::AbstractOperation::acceptsVectorInput ( int index = 0 ) const

inlinevirtual

Returns true if the operation can take a GL::Vector at a specific input.

Neural network operations may accept different kinds of data containers on inputs and outputs, namely Storage::View, GL::Vector and textures. This function is used to check whether a given operation accepts a vector on input.

Parameters

[in] index The input index. Expected to fall in the valid range, i.e. from zero to getInputCount() - 1 inclusive.

Reimplemented in Beatmup::NNets::Softmax, and Beatmup::NNets::Dense.

Definition at line 151 of file operation.h.

151 { return false; }

◆ acceptsTextureInput()

virtual bool Beatmup::NNets::AbstractOperation::acceptsTextureInput ( int index = 0 ) const

inlinevirtual

Returns true if the operation can take a GL::TextureHandler at a specific input.

Neural network operations may accept different kinds of data containers on inputs and outputs, namely Storage::View, GL::Vector and textures. This function is used to check whether a given operation accepts a texture on input.

Parameters

[in] index The input index. Expected to fall in the valid range, i.e. from zero to getInputCount() - 1 inclusive.

Reimplemented in Beatmup::NNets::ImageSampler, and Beatmup::NNets::Conv2D.

Definition at line 159 of file operation.h.

159 { return false; }

◆ acceptsStorageOutput()

virtual bool Beatmup::NNets::AbstractOperation::acceptsStorageOutput ( int index = 0 ) const

inlinevirtual

Returns true if the operation can take a Storage::View at a specific output.

Neural network operations may accept different kinds of data containers on outputs and outputs, namely Storage::View, GL::Vector and textures. This function is used to check whether a given operation accepts a storage view on output.

Parameters

[in] index The output index. Expected to fall in the valid range, i.e. from zero to getOutputCount() - 1 inclusive.

Reimplemented in Beatmup::NNets::Pooling2D, and Beatmup::NNets::Conv2D.

Definition at line 167 of file operation.h.

167 { return false; }

◆ acceptsVectorOutput()

virtual bool Beatmup::NNets::AbstractOperation::acceptsVectorOutput ( int index = 0 ) const

inlinevirtual

Returns true if the operation can take a GL::Vector at a specific output.

Neural network operations may accept different kinds of data containers on outputs and outputs, namely Storage::View, GL::Vector and textures. This function is used to check whether a given operation accepts a vector on output.

Parameters

[in] index The output index. Expected to fall in the valid range, i.e. from zero to getOutputCount() - 1 inclusive.

Reimplemented in Beatmup::NNets::Dense.

Definition at line 175 of file operation.h.

175 { return false; }

◆ acceptsTextureOutput()

virtual bool Beatmup::NNets::AbstractOperation::acceptsTextureOutput ( int index = 0 ) const

inlinevirtual

Returns true if the operation can take a GL::TextureHandler at a specific output.

Neural network operations may accept different kinds of data containers on outputs and outputs, namely Storage::View, GL::Vector and textures. This function is used to check whether a given operation accepts a texture on output.

Parameters

[in] index The output index. Expected to fall in the valid range, i.e. from zero to getOutputCount() - 1 inclusive.

Reimplemented in Beatmup::NNets::ImageSampler.

Definition at line 183 of file operation.h.

183 { return false; }

◆ getOutputSize()

Size AbstractOperation::getOutputSize ( int outputIndex = 0 ) const

virtual

Returns full size of a specific operation output.

Reimplemented in Beatmup::NNets::Pooling2D, Beatmup::NNets::ImageSampler, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

Definition at line 32 of file operation.cpp.

                                                            {
     throw RuntimeError("Operation " + name + " does not have output #" + std::to_string(outputIndex));
 }

◆ getOutput() [1/3]

Storage::View AbstractOperation::getOutput ( int index = 0 )

virtual

Returns a storage view bound to a specific operation output.

If no view is bound, returns empty view.

Parameters

[in] index The output index. Expected to fall in the valid range, i.e. from zero to getOutputCount() - 1 inclusive.

Reimplemented in Beatmup::NNets::Pooling2D, and Beatmup::NNets::Conv2D.

Definition at line 37 of file operation.cpp.

                                                   {
     throw RuntimeError("Operation " + name + " does not take Storage View on output #" + std::to_string(index));
 }

◆ getOutput() [2/3]

void AbstractOperation::getOutput	(	GL::Vector *&	vector,
		int	index = `0`
	)

virtual

Returns a GL::Vector bound to a specific operation output.

Parameters

[out]	vector	Pointer to the GL::Vector. If no vector is bound, becomes null.
[in]	index	The output index. Expected to fall in the valid range, i.e. from zero to getOutputCount() - 1 inclusive.

Reimplemented in Beatmup::NNets::Dense.

Definition at line 42 of file operation.cpp.

                                                        {
     throw RuntimeError("Operation " + name + " does not take Vector on output #" + std::to_string(index));
 }

◆ getOutput() [3/3]

void AbstractOperation::getOutput	(	GL::TextureHandler *&	vector,
		int	index = `0`
	)

virtual

Returns a GL::TextureHandler bound to a specific operation output.

Parameters

[out]	vector	Pointer to the GL::TextureHandler. If no texture is bound, becomes null.
[in]	index	The output index. Expected to fall in the valid range, i.e. from zero to getOutputCount() - 1 inclusive.

Reimplemented in Beatmup::NNets::ImageSampler.

Definition at line 47 of file operation.cpp.

                                                                {
     throw RuntimeError("Operation " + name + " does not take TextureHandler on output #" + std::to_string(index));
 }

◆ setInput() [1/3]

void AbstractOperation::setInput	(	Storage::View &&	storage,
		int	index = `0`
	)

virtual

Reimplemented in Beatmup::NNets::Softmax, Beatmup::NNets::Dense, Beatmup::NNets::Pooling2D, and Beatmup::NNets::Conv2D.

Definition at line 52 of file operation.cpp.

                                                          {
     throw RuntimeError("Operation " + name + " does not take Storage View on input #" + std::to_string(index));
 }

◆ setOutput() [1/3]

void AbstractOperation::setOutput	(	Storage::View &&	storage,
		int	index = `0`
	)

virtual

Reimplemented in Beatmup::NNets::Pooling2D, and Beatmup::NNets::Conv2D.

Definition at line 57 of file operation.cpp.

                                                           {
     throw RuntimeError("Operation " + name + " does not take Storage View on output #" + std::to_string(index));
 }

◆ setInput() [2/3]

void AbstractOperation::setInput	(	GL::Vector &	vector,
		int	index = `0`
	)

virtual

Reimplemented in Beatmup::NNets::Softmax, and Beatmup::NNets::Dense.

Definition at line 62 of file operation.cpp.

                                                      {
     throw RuntimeError("Operation " + name + " does not take Vector on input #" + std::to_string(index));
 }

◆ setOutput() [2/3]

void AbstractOperation::setOutput	(	GL::Vector &	vector,
		int	index = `0`
	)

virtual

Reimplemented in Beatmup::NNets::Dense.

Definition at line 67 of file operation.cpp.

                                                       {
     throw RuntimeError("Operation " + name + " does not take Vector on output #" + std::to_string(index));
 }

◆ setInput() [3/3]

void AbstractOperation::setInput	(	GL::TextureHandler &	image,
		int	index = `0`
	)

virtual

Reimplemented in Beatmup::NNets::ImageSampler, and Beatmup::NNets::Conv2D.

Definition at line 72 of file operation.cpp.

                                                                    {
     throw RuntimeError("Operation " + name + " does not take TextureHandler on input #" + std::to_string(index));
 }

◆ setOutput() [3/3]

void AbstractOperation::setOutput	(	GL::TextureHandler &	image,
		int	index = `0`
	)

virtual

Reimplemented in Beatmup::NNets::ImageSampler.

Definition at line 77 of file operation.cpp.

                                                               {
     throw RuntimeError("Operation " + name + " does not take TextureHandler on output #" + std::to_string(index));
 }

◆ serialize()

virtual std::map<std::string, std::string> Beatmup::NNets::AbstractOperation::serialize ( ) const

pure virtual

Returns a serialized representation of th operation;.

Implemented in Beatmup::NNets::Softmax, Beatmup::NNets::Pooling2D, Beatmup::NNets::ImageSampler, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

◆ disconnect()

virtual void Beatmup::NNets::AbstractOperation::disconnect ( )

pure virtual

Assigns empty inputs and outputs.

Implemented in Beatmup::NNets::Softmax, Beatmup::NNets::Pooling2D, Beatmup::NNets::ImageSampler, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

◆ countMultiplyAdds()

virtual unsigned long Beatmup::NNets::AbstractOperation::countMultiplyAdds ( ) const

inlinevirtual

Counts (approximate) number of multiply-adds used by this operation.

A single multiply-add is one multiplication and one addition.

Reimplemented in Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

Definition at line 232 of file operation.h.

232 { return 0; }

◆ countTexelFetches()

virtual unsigned long Beatmup::NNets::AbstractOperation::countTexelFetches ( ) const

inlinevirtual

Counts (approximate) number of texels fetches.

Reimplemented in Beatmup::NNets::Pooling2D, Beatmup::NNets::ImageSampler, Beatmup::NNets::Dense, and Beatmup::NNets::Conv2D.

Definition at line 237 of file operation.h.

237 { return 0; }

◆ getName()

std::string Beatmup::NNets::AbstractOperation::getName ( ) const

inline

Returns: operation name

Definition at line 242 of file operation.h.

242 { return name; }

Friends And Related Function Documentation

◆ Model

friend class Model

friend

Definition at line 47 of file operation.h.

Member Data Documentation

◆ name

std::string Beatmup::NNets::AbstractOperation::name

private

Definition at line 50 of file operation.h.

The documentation for this class was generated from the following files:

core/nnets/operation.h
core/nnets/operation.cpp

Classes

Public Member Functions

Protected Member Functions

Private Member Functions

Private Attributes

Friends

Detailed Description

Constructor & Destructor Documentation

◆ AbstractOperation() [1/2]

◆ AbstractOperation() [2/2]

◆ ~AbstractOperation()

Member Function Documentation

◆ prepare()

◆ execute() [1/2]

◆ execute() [2/2]

◆ getInputPadding()

◆ getSampledChannels()

◆ usesGpu()

◆ getInputCount()

◆ getOutputCount()

◆ acceptsStorageInput()

◆ acceptsVectorInput()

◆ acceptsTextureInput()

◆ acceptsStorageOutput()

◆ acceptsVectorOutput()

◆ acceptsTextureOutput()

◆ getOutputSize()

◆ getOutput() [1/3]

◆ getOutput() [2/3]

◆ getOutput() [3/3]

◆ setInput() [1/3]

◆ setOutput() [1/3]

◆ setInput() [2/3]

◆ setOutput() [2/3]

◆ setInput() [3/3]

◆ setOutput() [3/3]

◆ serialize()

◆ disconnect()

◆ countMultiplyAdds()

◆ countTexelFetches()

◆ getName()

Friends And Related Function Documentation

◆ Model

Member Data Documentation

◆ name