Beatmup::NNets::Model Class Reference

Neural net model. More...

#include <model.h>

Inheritance diagram for Beatmup::NNets::Model:

Classes
struct	Connection
	Connection descriptor. More...
struct	UserOutput
	A user-defined output descriptor. More...

Public Member Functions
	Model (Context &context, std::initializer_list< AbstractOperation * > ops)
	Instantiates a model from a list of operations interconnecting them in a feedforward fashion. More...
	Model (Context &context)
	Instantiates an empty model. More...
	~Model ()
void	append (AbstractOperation *newOp, bool connect=false)
	Adds a new operation to the model. More...
void	append (std::initializer_list< AbstractOperation * > newOps, bool connect=false)
	Adds new operations to the model. More...
void	addOperation (const std::string &opName, AbstractOperation *newOp)
	Adds a new operation to the model before another operation in the execution order. More...
void	addOperation (const AbstractOperation &operation, AbstractOperation *newOp)
void	addConnection (const std::string &sourceOpName, const std::string &destOpName, int output=0, int input=0, int shuffle=0)
	Adds a connection between two given ops. More...
void	addOutput (const std::string &operation, int output=0)
	Enables reading output data from the model memory through getOutputData(). More...
void	addOutput (const AbstractOperation &operation, int output=0)
const float *	getOutputData (size_t &numSamples, const std::string &operation, int output=0) const
	Reads data from the model memory. More...
const float *	getOutputData (size_t &numSamples, const AbstractOperation &operation, int output=0) const
virtual void	prepare (GraphicPipeline &gpu, ChunkCollection &data)
	Prepares all operations: reads the model data from chunks and builds GPU programs. More...
bool	isReady () const
void	execute (TaskThread &thread, GraphicPipeline *gpu)
	Runs the inference. More...
bool	isOperationInModel (const AbstractOperation &operation) const
	Checks if a specific operation makes part of the model. More...
AbstractOperation &	getFirstOperation ()
AbstractOperation &	getLastOperation ()
const AbstractOperation &	getFirstOperation () const
const AbstractOperation &	getLastOperation () const
size_t	getNumberOfOperations () const
template<class OperationClass = AbstractOperation>
OperationClass &	getOperation (const std::string &operationName)
	Retrieves an operation by its name. More...
const ProgressTracking &	getPreparingProgress () const
	Returns model preparation progress tracking. More...
const ProgressTracking &	getInferenceProgress () const
	Returns inference progress tracking. More...
unsigned long	countMultiplyAdds () const
	Provides an estimation of the number of multiply-adds characterizing the model complexity. More...
unsigned long	countTexelFetches () const
	Provides an estimation of the total number of texels fetched by all the operations in the model per image. More...
size_t	getMemorySize () const
	Returns the amount of texture memory in bytes currently allocated by the model to run the inference. More...
Listing	serialize () const
	Returns serialized representation of the model as a Listing. More...
std::string	serializeToString () const
	Returns serialized representation of the model as a string. More...
void	setProfiler (Profiler *profiler)
	Attaches a profiler instance to meter the execution time per operation during the inference. More...
Public Member Functions inherited from Beatmup::GL::ProgramBank
	ProgramBank (Context &context)
	~ProgramBank ()
GL::RenderingProgram *	operator() (GraphicPipeline &gpu, const std::string &code, bool enableExternalTextures=false)
	Provides a program given a fragment shader source code. More...
void	release (GraphicPipeline &gpu, GL::RenderingProgram *program)
	Marks a program as unused any more. More...
Public Member Functions inherited from Beatmup::Object
virtual	~Object ()

Protected Member Functions
void	freeMemory ()
	Frees all allocated storages. More...
Storage &	allocateStorage (GraphicPipeline &gpu, const Size size, bool forGpu=true, bool forCpu=false, const int pad=0, const int reservedChannels=0)
	Allocates a new storage. More...
Storage &	allocateFlatStorage (GraphicPipeline &gpu, const int size)
	Allocates a new flat storage. More...
GL::Vector &	allocateVector (GraphicPipeline &gpu, const int size)
	Allocates a vector that can be used as operation input or output. More...
InternalBitmap &	allocateTexture (GraphicPipeline &gpu, const Size size)
	Allocates a texture that can be used as operation input or output. More...
bool	isPreceding (const AbstractOperation &first, const AbstractOperation &second) const
	Checks whether an operation goes before another operation in the model according the ops execution order. More...
AbstractOperation *	operator[] (const std::string &operationName)
const AbstractOperation *	operator[] (const std::string &operationName) const
void	addConnection (AbstractOperation &source, AbstractOperation &dest, int output=0, int input=0, int shuffle=0)

Protected Attributes
std::vector< AbstractOperation * >	ops
	model operations More...
ProgressTracking	preparingProgress
	model preparation progress More...
ProgressTracking	inferenceProgress
	inference progress More...
bool	ready
	if true, ops are connected to each other and storages are allocated More...
Protected Attributes inherited from Beatmup::GL::ProgramBank
Context &	context

Private Attributes
std::multimap< const AbstractOperation *, Connection >	connections
	source operation => connection descriptor mapping More...
std::multimap< const AbstractOperation *, UserOutput >	userOutputs
	operation => user output mapping More...
std::vector< Storage * >	storages
	allocated storages used during the inference More...
std::vector< GL::Vector * >	vectors
	allocated vectors used during the inference More...
std::vector< InternalBitmap * >	textures
	allocated images used during the inference More...
Profiler *	profiler
	pointer to a Profiler attached to the model More...

Detailed Description

Neural net model.

Contains a list of operations and programmatically defined interconnections between them using addConnection(). Enables access to the model memory at any point in the model through addOutput() and getOutputData(). The memory needed to store internal data during the inference is allocated automatically; storages are reused when possible. The inference of a Model is performed by InferenceTask.

Definition at line 92 of file model.h.

Constructor & Destructor Documentation

Model() [1/2]

Model::Model	(	Context &	context,
		std::initializer_list< AbstractOperation * >	ops
	)

Instantiates a model from a list of operations interconnecting them in a feedforward fashion.

The first output of every operation is connected to the first input of its successor. Optional connections may be added after model creation.

Parameters

[in,out]	context	A context instance
[in]	ops	Operations given in the execution order. The Model does not take ownership of them.

Definition at line 27 of file model.cpp.

                                                                         :
    ProgramBank(context),
    profiler(nullptr), ready(false),
    ops(ops.begin(), ops.end())
{
    // establish feedforward connections
    for (size_t i = 1; i < this->ops.size(); ++i)
        addConnection(*this->ops[i - 1], *this->ops[i]);
}

Model() [2/2]

Model::Model

(

Context &

context

)

Instantiates an empty model.

Parameters

[in,out]

context

A context instance used to store internal resources needed for inference

Definition at line 38 of file model.cpp.

: Model(context, {}) {}

~Model()

Model::~Model

(

)

Definition at line 40 of file model.cpp.

              {
    for (auto op : ops)
        op->disconnect();
    freeMemory();
}

Member Function Documentation

freeMemory()

void Model::freeMemory ( )

protected

Frees all allocated storages.

Definition at line 415 of file model.cpp.

                       {
    for (auto storage : storages)
        delete storage;
    storages.clear();
    for (auto vector : vectors)
        delete vector;
    vectors.clear();
    for (auto texture : textures)
        delete texture;
    textures.clear();
}

allocateStorage()

Storage & Model::allocateStorage ( GraphicPipeline &  gpu, const Size  size, bool  forGpu = true, bool  forCpu = false, const int  pad = 0, const int  reservedChannels = 0  )

protected

Allocates a new storage.

Its views might be used as operations inputs and outputs. The storage is destroyed together with the model.

Parameters

[in,out]	gpu	A graphic pipeline instance
[in]	size	The storage size (width, height, number of channels)
[in]	forGpu	Allocate for the use on GPU
[in]	forCpu	Allocate for the use on CPU
[in]	pad	Storage padding: number of pixels added on both sides along width and height of every channel
[in]	reservedChannels	Number of additional channels that may be sampled together with the storage. This does not change the storage size, but impacts the way the channels are packed into the textures. It allows the storage to be sampled with other storages of a specific total depth in the same shader, if the addDepth is greater or equal to the total depth.

Returns

newly allocated storage.

Definition at line 428 of file model.cpp.

                                                                                                                                       {
    Storage* storage = new Storage(context, gpu, size, pad, reservedDepth);
    if (forGpu)
        storage->allocate(gpu);
    if (forCpu)
        storage->allocate();
    storages.push_back(storage);
    return *storage;
}

allocateFlatStorage()

Storage & Model::allocateFlatStorage ( GraphicPipeline &  gpu, const int  size  )

protected

Allocates a new flat storage.

Its views are be used as operations inputs and outputs. Flat storages can be inputs of Dense layers. The storage is destroyed together with the model.

Parameters

[in,out]	gpu	A graphic pipeline instance
[in]	size	Number of samples in the storage

Returns

newly allocated storage.

Definition at line 439 of file model.cpp.

                                                                  {
    Storage* storage = new Storage(context, gpu, Size(1, 1, size));
    storage->allocate(gpu);
    storages.push_back(storage);
    return *storage;
}

allocateVector()

GL::Vector & Model::allocateVector ( GraphicPipeline &  gpu, const int  size  )

protected

Allocates a vector that can be used as operation input or output.

Differently to flat storages, vectors store floating point data (GL ES 3.1 and higher) or 16-bit signed fixed point values with 8 bits fractional part (GL ES 2.0).

Parameters

[in,out]	gpu	A graphic pipeline instance
[in]	size	Number of samples in the vector

Definition at line 447 of file model.cpp.

                                                                    {
    GL::Vector::Format format;
#ifdef BEATMUP_OPENGLVERSION_GLES20
    format = GL::Vector::Format::FIXED16;
#else
    format = GL::Vector::Format::FLOAT;
#endif
    GL::Vector* vector = new GL::Vector(context, gpu, size, format);
    vectors.push_back(vector);
    return *vector;
}

allocateTexture()

InternalBitmap & Model::allocateTexture ( GraphicPipeline &  gpu, const Size  size  )

protected

Allocates a texture that can be used as operation input or output.

Parameters

[in,out]	gpu	A graphic pipeline instance
[in]	size	Image size. The depth can be 1, 3 or 4 channels.

Definition at line 460 of file model.cpp.

                                                                            {
    PixelFormat pixelFormat(PixelFormat::TripleByte);
    switch (size.getDepth()) {
    case 1:
        pixelFormat = PixelFormat::SingleByte;
        break;
    case 3:
        pixelFormat = PixelFormat::TripleByte;
        break;
    case 4:
        pixelFormat = PixelFormat::QuadByte;
        break;
    default:
        throw InvalidArgument("Unsupported depth: " + std::to_string(size.getDepth()));
    }
    textures.push_back(new InternalBitmap(context, pixelFormat, size.getWidth(), size.getHeight()));
    return *textures.back();
}

isPreceding()

bool Model::isPreceding ( const AbstractOperation &  first, const AbstractOperation &  second  ) const

protected

Checks whether an operation goes before another operation in the model according the ops execution order.

Parameters

[in]	first	The first operation (expected to be executed earlier)
[in]	second	The first operation (expected to be executed later)

Returns

true if both operations are in the model, and the first one is executed before the second one, false otherwise.

Definition at line 480 of file model.cpp.

                                                                                             {
    for (size_t firstIdx = 0; firstIdx < ops.size(); ++firstIdx)
        if (ops[firstIdx] == &first) {
            for (size_t secondIdx = firstIdx + 1; secondIdx < ops.size(); ++secondIdx)
                if (ops[secondIdx] == &second)
                    return true;
            return false;
        }
    return false;
}

operator[]() [1/2]

AbstractOperation * Model::operator[] ( const std::string &  operationName )

protected

Definition at line 492 of file model.cpp.

                                                                   {
    for (auto op : ops)
        if (op->getName() == operationName)
            return op;
    throw InvalidArgument("Operation not found: " + operationName);
}

operator[]() [2/2]

const AbstractOperation * Model::operator[] ( const std::string &  operationName ) const

protected

Definition at line 500 of file model.cpp.

                                                                               {
    for (auto op : ops)
        if (op->getName() == operationName)
            return op;
    throw InvalidArgument("Operation not found: " + operationName);
}

addConnection() [1/2]

void Model::addConnection ( AbstractOperation &  source, AbstractOperation &  dest, int  output = 0, int  input = 0, int  shuffle = 0  )

protected

Definition at line 91 of file model.cpp.

                                                                                                                {
    RuntimeError::check(0 <= output && output < source.getOutputCount(),
        "Operation " + source.getName() + " does not have output #" + std::to_string(output));
    RuntimeError::check(0 <= input && input < dest.getInputCount(),
        "Operation " + dest.getName() + " does not have input #" + std::to_string(input));
    connections.emplace(&source, Connection{ &dest, output, input, shuffle });
    ready = false;
}

append() [1/2]

void Model::append	(	AbstractOperation *	newOp,
		bool	connect = false
	)

Adds a new operation to the model.

The operation is added to the end of the operations list. The execution order corresponds to the addition order. The Model does not takes ownership of the passed pointer.

Parameters

[in]	newOp	The new operation
[in]	connect	If true, the main operation input (#0) is connected to the main output (#0) of the last operation

Definition at line 47 of file model.cpp.

                                                         {
    for (auto op : ops) {
        if (op == newOp)
            throw RuntimeError("Cannot add operation " + newOp->getName() + " to the model: already added");
        else
            if (op->getName() == newOp->getName())
                throw RuntimeError("Cannot add operation " + newOp->getName() + " to the model: an operation with the same exists in the model");
    }
    ops.push_back(newOp);
    if (connect)
        addConnection(*ops[ops.size() - 2], *ops.back(), 0, 0, 0);
    ready = false;
}

append() [2/2]

void Model::append	(	std::initializer_list< AbstractOperation * >	newOps,
		bool	connect = false
	)

Adds new operations to the model.

The operations are added to the end of the operations list. The execution order corresponds to the addition order. The Model does not takes ownership of the passed pointer.

Parameters

[in]	newOps	The new operations
[in]	connect	If true, the main input (#0) of every operation is connected to the main output (#0) of the preceding operation

Definition at line 62 of file model.cpp.

                                                                               { 
    for (auto op : newOps)
        append(op, connect);
}

addOperation() [1/2]

void Model::addOperation	(	const std::string &	opName,
		AbstractOperation *	newOp
	)

Adds a new operation to the model before another operation in the execution order.

The Model does not takes ownership of the passed pointer. The new operation is not automatically connected to other operations.

Parameters

[in]	opName	Name of the operation the new operation is inserted before
[in]	newOp	The new operation

Definition at line 68 of file model.cpp.

                                                                          {
    auto it = std::find_if(ops.begin(), ops.end(), [&opName](AbstractOperation* op){ return op->getName() == opName; });
    if (it == ops.end())
        throw InvalidArgument("Cannot find operation " + opName);
    ops.insert(it, newOp);
}

addOperation() [2/2]

void Model::addOperation	(	const AbstractOperation &	operation,
		AbstractOperation *	newOp
	)

Definition at line 76 of file model.cpp.

                                                                              {
    auto it = std::find(ops.begin(), ops.end(), &op);
    if (it == ops.end())
        throw InvalidArgument("Operation " + op.getName() + " is not in the model");
    ops.insert(it, newOp);
}

addConnection() [2/2]

void Model::addConnection	(	const std::string &	sourceOpName,
		const std::string &	destOpName,
		int	output = 0,
		int	input = 0,
		int	shuffle = 0
	)

Adds a connection between two given ops.

Parameters

[in]	sourceOpName	Name of the operation emitting the data
[in]	destOpName	Name of the operation receiving the data
[in]	output	Output number of the source operation
[in]	input	Input number of the destination operation
[in]	shuffle	If greater than zero, the storage is shuffled. For shuffle = n, the output channels are sent to the destination operation in the following order: 0, 1, 2, 3, 4n, 4n+1, 4n+2, 4n+3, 8n, 8n+1, 8n+2, 8n+3, ..., 4, 5, 6, 7, 4n+4, 4n+5, 4n+6, 4n+7, 8n+4, ...

Definition at line 84 of file model.cpp.

                                                                                                                        {
    auto& source = getOperation(sourceOpName);
    auto& dest = getOperation(destOpName);
    addConnection(source, dest, output, input, shuffle);
}

addOutput() [1/2]

void Model::addOutput	(	const std::string &	operation,
		int	output = 0
	)

Enables reading output data from the model memory through getOutputData().

A given operation output is connected to a storage that might be accessed by the application after the run.

Parameters

[in]	operation	Name of the operation or the operation itself to get data from
[in]	output	The operation output index

Definition at line 101 of file model.cpp.

                                                         {
    auto op = (*this)[opName];
    auto outputs = userOutputs.equal_range(op);
    for (auto i = outputs.first; i != outputs.second; ++i)
        if (i->second.index == output)
            // already added
            return;
    userOutputs.emplace(op, UserOutput{ output });
    ready = false;
}

addOutput() [2/2]

void Model::addOutput	(	const AbstractOperation &	operation,
		int	output = 0
	)

Definition at line 113 of file model.cpp.

                                                                    {
    RuntimeError::check(isOperationInModel(operation), "Operation " + operation.getName() + " is not in the model");
    auto outputs = userOutputs.equal_range(&operation);
    for (auto i = outputs.first; i != outputs.second; ++i)
        if (i->second.index == output)
            // already added
            return;
    userOutputs.emplace(&operation, UserOutput{ output });
    ready = false;
}

getOutputData() [1/2]

const float * Model::getOutputData	(	size_t &	numSamples,
		const std::string &	operation,
		int	output = 0
	)		const

Reads data from the model memory.

addOutput() is needed to be called first in order to enable reading the data. Otherwise null is returned.

Parameters

[out]	numSamples	Returns number of samples in the pointed data buffer
[in]	operation	Name of the operation or the operation itself to get data from
[in]	output	The operation output index

Returns

pointer to the data stored as a 3D array of (height, width, channels) layout, or null.

Definition at line 125 of file model.cpp.

                                                                                                  {
    return getOutputData(numSamples, *(*this)[operation], output);
}

getOutputData() [2/2]

const float * Model::getOutputData	(	size_t &	numSamples,
		const AbstractOperation &	operation,
		int	output = 0
	)		const

Definition at line 130 of file model.cpp.

                                                                                                          {
    auto outputs = userOutputs.equal_range(&operation);
    for (auto i = outputs.first; i != outputs.second; ++i)
        if (i->second.index == output) {
            numSamples = i->second.data.size();
            return i->second.data.data();
        }
 
    numSamples = 0;
    return nullptr;
}

prepare()

void Model::prepare ( GraphicPipeline &  gpu, ChunkCollection &  data  )

virtual

Prepares all operations: reads the model data from chunks and builds GPU programs.

The inputs of the model needed to be provided. Preparation progress is tracked by a ProgressTracking instance (getPreparingProgress()).

Parameters

[in,out]	gpu	A graphic pipeline instance
[in]	data	ChunkCollection containing the model data

Definition at line 143 of file model.cpp.

                                                                {
    if (ready)
        return;
    freeMemory();
 
    std::map<Storage*, std::vector<AbstractOperation*>> refs;
        // Contains ops that use a specific storage as input, meaning that it cannot be reused elsewhere.
        // If no ops refer a storage, in can be recycled.
 
    // find input depth capping
    // If too many channels are sampled by an op having multiple inputs, its input storages will have reserved channels.
    const int sampledChannelsLimit = 4 * gpu.getLimit(GraphicPipeline::Limit::TEXTURE_IMAGE_UNITS);
    std::map<AbstractOperation*, int> sampledChannels;   // op => number of sampled channels
    for (auto conn : connections) {
        auto* op = conn.second.dest;
        // get the number of sampled channels
        int min, max;
        op->getSampledChannels(conn.second.input, min, max);
        // cap the maximum: a storage will not have more channels than the limit anyway
        max = std::min(max, sampledChannelsLimit);
        // add to input channels
        sampledChannels[op] += max;
    }
 
    // loop through connected ops
    data.open();
    preparingProgress.reset(ops.size());
    for (auto src : ops) {
        std::vector<Beatmup::Object*> outputs(src->getOutputCount(), nullptr);  // src output index => storage/vector bound to the output
        std::vector<int> paddings(src->getOutputCount(), 0);    // src output index => max padding over all connections
        Bitset connectedOutputs(src->getOutputCount(), false);
 
        // loop over connections to find max paddings per output
        auto connections = this->connections.equal_range(src);
        for (auto i = connections.first; i != connections.second; ++i) {
            const auto& conn = i->second;
            paddings[conn.output] = std::max(paddings[conn.output], conn.dest->getInputPadding(conn.input));
        }
 
        // loop over connections
        for (auto i = connections.first; i != connections.second; ++i) {
            const auto& conn = i->second;
            auto* dst = conn.dest;
            connectedOutputs.set(conn.output);
 
            if (outputs[conn.output])
                RuntimeError::check(src->acceptsStorageOutput(conn.output) ^ src->acceptsVectorOutput(conn.output) ^ src->acceptsTextureOutput(conn.output),
                    "Operation output accepting different types can only have a single connection");
                    // To avoid output type mismatch when connecting second time
 
            // if a regular Storage is accepted by both source and destination
            if (src->acceptsStorageOutput(conn.output) && dst->acceptsStorageInput(conn.input)) {
                const Size size = src->getOutputSize(conn.output);
                Storage* storage = nullptr;
 
                // check if the output storage is already allocated
                if (outputs[conn.output]) {
                    storage = static_cast<Storage*>(outputs[conn.output]);
                    refs[storage].push_back(dst);
                }
 
                else {
                    // decide on reserved depth (if capping)
                    int depthCapping = 0;
                    if (sampledChannels[dst] > sampledChannelsLimit) {
                        // the op exceeds the limit
                        int min, max;
                        dst->getSampledChannels(conn.input, min, max);
                        const int cappingMargin = std::min(sampledChannelsLimit, size[2]) - min;    // this is how much we can cap at the current input
                        if (cappingMargin > 0) {
                            depthCapping = std::min(cappingMargin, sampledChannels[dst] - sampledChannelsLimit);
                            // reduce the excess
                            sampledChannels[dst] -= depthCapping;
                        }
                    }
 
                    // try to recycle an existing storage first
                    for (auto& i : refs) {
                        auto candidate = i.first;
                        auto& users = i.second;
                        const int reservedDepth = sampledChannelsLimit - 4 * candidate->getNumberOfTextures();
                        // check if (1) size matches, (2) padding is sufficient, (3) reserved depth matches the number of channels to cap or no capping
                        if (candidate->getSize() == size && candidate->getPadding() >= dst->getInputPadding(conn.input) && (reservedDepth == depthCapping || depthCapping == 0)
                            && users.empty())
                        {
                            // found!
                            storage = candidate;
                            users.push_back(dst);
                            break;
                        }
                        if (storage)
                            break;
                    }
 
                    // no matching storage found, allocate a new one
                    if (!storage) {
                        storage = (size[0] == 1 && size[1] == 1) ?
                            // allocate flat storage if the output size is of 1x1 pixels
                            &allocateFlatStorage(gpu, size[2]) :
                            &allocateStorage(gpu,
                                size,
                                src->usesGpu(), !src->usesGpu(),
                                paddings[conn.output],
                                depthCapping
                            );
                        refs.emplace(storage, std::vector<AbstractOperation*>{ dst });
                    }
 
                    // mark output as allocated
                    outputs[conn.output] = storage;
                }
 
                // connect
                src->setOutput(*storage, conn.output);
                if (conn.shuffle > 0)
                    dst->setInput(Storage::View(*storage, conn.shuffle), conn.input);
                else
                    dst->setInput(*storage, conn.input);
            }
 
            // if a Vector is accepted
            else if (src->acceptsVectorOutput(conn.output) && dst->acceptsVectorInput(conn.input)) {
                RuntimeError::check(conn.shuffle == 0, "Cannot shuffle vector");
                GL::Vector* vector;
 
                // check if the output storage is already allocated
                if (outputs[conn.output])
                    vector = static_cast<GL::Vector*>(outputs[conn.output]);
                else {
                    vector = &allocateVector(gpu, src->getOutputSize(conn.output).volume());
                    outputs[conn.output] = vector;
                }
 
                // connect
                src->setOutput(*vector, conn.output);
                dst->setInput(*vector, conn.input);
            }
 
            // if a texture is accepted
            else if (src->acceptsTextureOutput(conn.output) && dst->acceptsTextureInput(conn.input)) {
                RuntimeError::check(conn.shuffle == 0, "Cannot shuffle texture");
                InternalBitmap* texture;
 
                // check if the output storage is already allocated
                if (outputs[conn.output])
                    texture = static_cast<InternalBitmap*>(outputs[conn.output]);
                else
                    outputs[conn.output] = texture = &allocateTexture(gpu, src->getOutputSize(conn.output));
 
                // connect
                src->setOutput(*texture, conn.output);
                dst->setInput(*texture, conn.input);
            }
 
            else
                throw RuntimeError("Cannot connect " + src->getName() + " (output #" + std::to_string(conn.output) + ") "
                    "to " + dst->getName() + " (input #" + std::to_string(conn.input) + "): storage type mismatch");
        }
 
        // allocate user outputs if not yet
        auto userOutputs = this->userOutputs.equal_range(src);
        for (auto i = userOutputs.first; i != userOutputs.second; ++i) {
            int idx = i->second.index;
            if (idx >= src->getOutputCount())
                throw InvalidArgument("Operation " + src->getName() + " does not have output #" + std::to_string(idx));
            if (!connectedOutputs[idx])
                if (src->acceptsStorageOutput(idx)) {
                    src->setOutput(allocateStorage(gpu, src->getOutputSize(idx), src->usesGpu(), !src->usesGpu()), idx);
                }
                else if (src->acceptsVectorOutput(idx)) {
                    src->setOutput(allocateVector(gpu, src->getOutputSize(idx).volume()), idx);
                }
        }
 
        // prepare operation
        src->prepare(gpu, data, *this);
 
        // remove references to storages used by the current operation. This allows their reuse in other connections.
        for (auto& i : refs) {
            auto& users = i.second;
            for (auto op = users.begin(); op != users.end(); )
                if (*op == src)
                    users.erase(op);
                else
                    ++op;
        }
 
        // advance the progress bar
        preparingProgress();
    }
 
    data.close();
    ready = true;
}

isReady()

bool Beatmup::NNets::Model::isReady ( ) const

inline

Returns

true if the model is ready to be used for inference (prepare() has been called).

Definition at line 278 of file model.h.

{ return ready; }

execute()

void Model::execute	(	TaskThread &	thread,
		GraphicPipeline *	gpu
	)

Runs the inference.

Parameters

[in,out]	thread	Task thread instance
[in,out]	gpu	A graphic pipeline

Definition at line 339 of file model.cpp.

                                                            {
    if (gpu)
        gpu->switchMode(GraphicPipeline::Mode::INFERENCE);
 
    // reset the progress tracker
    inferenceProgress.reset(ops.size());
 
    // loop through ops
    for (auto op : ops) {
        if (thread.isTaskAborted())
            return;
 
        // start profiling
        if (thread.isManaging() && profiler)
            (*profiler)(op->getName());
 
        // run operation
        try {
            if (gpu)
                op->execute(thread, *gpu);
            else
                op->execute(thread);
        } catch (const std::exception& ex) {
            throw InferenceTimeError(*op, ex);
        }
 
        // get user outputs
        auto userOutputs = this->userOutputs.equal_range(op);
        for (auto it = userOutputs.first; it != userOutputs.second; ++it) {
            int idx = it->second.index;
            auto& data = it->second.data;
            if (gpu)
                if (op->acceptsStorageOutput(idx)) {
                    // get data pointer from storage
                    auto view = op->getOutput(idx);
                    if (!view.getStorage().isUpToDate(ProcessingTarget::CPU))
                        view.getStorage().pull(*gpu);
 
                    // copy to the vector
                    Storage::Scanner scan(view);
                    scan.move(0, 0);
                    data.resize(view.getSize().volume());
                    for (auto it = data.begin(); it != data.end(); it += view.getDepth()) {
                        scan.fill(it, data.end());
                        ++scan;
                    }
                }
                else if (op->acceptsVectorOutput(idx)) {
                    GL::Vector* vector;
                    op->getOutput(vector, idx);
                    vector->fetch(*gpu, data);
                }
        }
 
        if (thread.isManaging()) {
            // stop profiler
            if (profiler) {
                gpu->flush();   // wait till GPU is done
                profiler->lap();
            }
 
            // increase inference progress
            inferenceProgress();
        }
    }
}

isOperationInModel()

bool Model::isOperationInModel

(

const AbstractOperation &

operation

)

const

Checks if a specific operation makes part of the model.

Returns

true if the operation is in the model.

Definition at line 407 of file model.cpp.

                                                                       {
    for (auto op : ops)
        if (op == &operation)
            return true;
    return false;
}

getFirstOperation() [1/2]

AbstractOperation& Beatmup::NNets::Model::getFirstOperation ( )

inline

Definition at line 293 of file model.h.

{ return *ops.front(); }

getLastOperation() [1/2]

AbstractOperation& Beatmup::NNets::Model::getLastOperation ( )

inline

Definition at line 294 of file model.h.

{ return *ops.back(); }

getFirstOperation() [2/2]

const AbstractOperation& Beatmup::NNets::Model::getFirstOperation ( ) const

inline

Definition at line 295 of file model.h.

{ return *ops.front(); }

getLastOperation() [2/2]

const AbstractOperation& Beatmup::NNets::Model::getLastOperation ( ) const

inline

Definition at line 296 of file model.h.

{ return *ops.back(); }

getNumberOfOperations()

size_t Beatmup::NNets::Model::getNumberOfOperations ( ) const

inline

Definition at line 297 of file model.h.

{ return ops.size(); }

getOperation()

template<class OperationClass = AbstractOperation>

OperationClass& Beatmup::NNets::Model::getOperation ( const std::string &  operationName )

inline

Retrieves an operation by its name.

Definition at line 303 of file model.h.

                                                                                {
                return *static_cast<OperationClass*>((*this)[operationName]);
            }

getPreparingProgress()

const ProgressTracking& Beatmup::NNets::Model::getPreparingProgress ( ) const

inline

Returns model preparation progress tracking.

Definition at line 310 of file model.h.

{ return preparingProgress; }

getInferenceProgress()

const ProgressTracking& Beatmup::NNets::Model::getInferenceProgress ( ) const

inline

Returns inference progress tracking.

Definition at line 315 of file model.h.

{ return inferenceProgress; }

countMultiplyAdds()

unsigned long Model::countMultiplyAdds

(

)

const

Provides an estimation of the number of multiply-adds characterizing the model complexity.

Queries the number of multiply-adds of every operation of the model and sums them up.

Definition at line 508 of file model.cpp.

                                             {
    unsigned long result = 0;
    for (auto op : ops)
        result += op->countMultiplyAdds();
    return result;
}

countTexelFetches()

unsigned long Model::countTexelFetches

(

)

const

Provides an estimation of the total number of texels fetched by all the operations in the model per image.

Definition at line 516 of file model.cpp.

                                             {
    unsigned long result = 0;
    for (auto op : ops)
        result += op->countTexelFetches();
    return result;
}

getMemorySize()

size_t Model::getMemorySize

(

)

const

Returns the amount of texture memory in bytes currently allocated by the model to run the inference.

When the model is ready to run, this represents the size of the memory needed to store internal data during the inference. The resulting value does not include the size of GLSL shaders binaries stored in GPU memory which can be significant.

Definition at line 524 of file model.cpp.

                                  {
    size_t size = 0;
    for (auto& entry : storages)
        size += entry->getMemorySize();
    for (auto& entry : vectors)
        size += entry->getMemorySize();
    for (auto& entry : textures)
        size += entry->getMemorySize();
    return size;
}

serialize()

Listing Beatmup::NNets::Model::serialize

(

)

const

Returns serialized representation of the model as a Listing.

serializeToString()

std::string Model::serializeToString

(

)

const

Returns serialized representation of the model as a string.

Definition at line 579 of file model.cpp.

                                         {
    Listing listing(serialize());
    std::stringstream strstr;
    listing.printOut(strstr);
    return strstr.str();
}

setProfiler()

void Beatmup::NNets::Model::setProfiler ( Profiler *  profiler )

inline

Attaches a profiler instance to meter the execution time per operation during the inference.

This may slow down the inference.

Parameters

[in]

profiler

A profiler instance or null pointer (to disable the profiling)

Definition at line 350 of file model.h.

{ this->profiler = profiler; }

Member Data Documentation

connections

std::multimap<const AbstractOperation*, Connection> Beatmup::NNets::Model::connections

private

source operation => connection descriptor mapping

Definition at line 113 of file model.h.

userOutputs

std::multimap<const AbstractOperation*, UserOutput> Beatmup::NNets::Model::userOutputs

private

operation => user output mapping

Definition at line 114 of file model.h.

storages

std::vector<Storage*> Beatmup::NNets::Model::storages

private

allocated storages used during the inference

Definition at line 116 of file model.h.

vectors

std::vector<GL::Vector*> Beatmup::NNets::Model::vectors

private

allocated vectors used during the inference

Definition at line 117 of file model.h.

textures

std::vector<InternalBitmap*> Beatmup::NNets::Model::textures

private

allocated images used during the inference

Definition at line 118 of file model.h.

profiler

Profiler* Beatmup::NNets::Model::profiler

private

pointer to a Profiler attached to the model

Definition at line 119 of file model.h.

ops

std::vector<AbstractOperation*> Beatmup::NNets::Model::ops

protected

model operations

Definition at line 122 of file model.h.

preparingProgress

ProgressTracking Beatmup::NNets::Model::preparingProgress

protected

model preparation progress

Definition at line 123 of file model.h.

inferenceProgress

ProgressTracking Beatmup::NNets::Model::inferenceProgress

protected

inference progress

Definition at line 124 of file model.h.

ready

bool Beatmup::NNets::Model::ready

protected

if true, ops are connected to each other and storages are allocated

Definition at line 125 of file model.h.

---

The documentation for this class was generated from the following files:

• core/nnets/model.h
• core/nnets/model.cpp