Beatmup::NNets::Storage Class Reference

3D tensor stored in a set of textures. More...

#include <storage.h>

Inheritance diagram for Beatmup::NNets::Storage:

Classes
class	Binder
	Binding of different input/output storages/texture handlers to a GLSL program. More...
class	InconsistentStorageState
class	Scanner
	Scans a storageview in RAM for further computations on CPU. More...
struct	Texture
class	TextureHandler
	TextureHandler representation of a pack of 4 channels from a non-empty View. More...
class	View
	Maps a 3D tensor onto a storage. More...

Public Member Functions
	Storage (Context &ctx, GraphicPipeline &gpu, const Size size, const int pad, const int reservedChannels=0)
	Creates a storage. More...
	Storage (Context &ctx, GraphicPipeline &gpu, const Size size)
	Creates a flat storage. More...
	~Storage ()
void	allocate (GraphicPipeline &gpu)
	Allocates the storage in GPU memory. More...
void	allocate ()
	Allocates the storage in RAM. More...
void	free (GraphicPipeline &gpu)
	Frees the allocated memory immediately. More...
void	free ()
	Deferred storage disposal: the textures are put into the GPU recycle bin associated with the context. More...
void	pull (GraphicPipeline &gpu)
	Pulls storage data from GPU memory to RAM. More...
void	push (GraphicPipeline &gpu)
	Pushes storage data from RAM to CPU memory. More...
void	push (GraphicPipeline &gpu, const float *hwcData, const size_t numSamples)
	Pushes a given data to GPU memory. More...
bool	isUpToDate (ProcessingTarget target) const
	Checks if the storage is up to date for a given processing target (CPU or GPU). More...
bool	isAllocated () const
	Returns true if the storage is allocated. More...
InternalBitmap *	getImage (Context &ctx, GraphicPipeline &gpu, int channel) const
	Converts a feature channel into a bitmap for debugging purposes. More...
int	getNumberOfTextures () const
	Returns total number of textures in the storage. More...
int	getChannelTextureNumber (int channel) const
	Returns number of texture containing a given channel. More...
IntPoint	getChannelOrigin (int channel) const
	Returns origin in pixels of a given channel within the texture containing it. More...
int	getTextureWidth () const
	Returns width in pixels of all the textures. More...
int	getTextureHeight () const
	Returns height in pixels of all the textures. More...
int	getPadding () const
	Returns storage padding. More...
Size	getSize () const
	Returns storage size in pixels. More...
size_t	getMemorySize () const
AlignedMemory &	getMemory ()
	Returns the storage memory, if allocated in RAM (for CPU). More...
const AlignedMemory &	getMemory () const
Public Member Functions inherited from Beatmup::Object
virtual	~Object ()

Static Public Member Functions
static void	checkChannelNumber (int channel)
	Checks whether a channel number points to the first channel in a texture. More...

Private Member Functions
	Storage (const Storage &)=delete
	disabling copying constructor More...
void	push (GraphicPipeline &gpu, const void *data)

Private Attributes
Context &	context
Texture *	textures
AlignedMemory	memory
	data storage in RAM More...
const Size	size
const int	pad
	padding in pixels added along width and height dimensions More...
int	packX
int	packY
	number of blocks of 4 channels per texture (spatial packing) More...
bool	upToDate [2]

Friends
class	Binder
class	View

Detailed Description

3D tensor stored in a set of textures.

Constraints: The tensor entries are in [0, 1] range and sampled in 8 bits. The tensor depth is a multiple of 4. Feature maps are stored as RGBA textures (at least four channels per texture). More feature maps can be packed in the same texture. The textures are all of the same size though.

Definition at line 126 of file storage.h.

Constructor & Destructor Documentation

Storage() [1/3]

Beatmup::NNets::Storage::Storage ( const Storage &  )

privatedelete

disabling copying constructor

Storage() [2/3]

Storage::Storage	(	Context &	ctx,
		GraphicPipeline &	gpu,
		const Size	size,
		const int	pad,
		const int	reservedChannels = 0
	)

Creates a storage.

Parameters

[in]	ctx	A context
[in]	gpu	A graphic pipeline instance
[in]	size	Storage size
[in]	pad	Additional padding to add to the texture dimensions
[in]	reservedChannels	Number of depth dimensions that may be sampled together with this storage

Definition at line 127 of file storage.cpp.

                                                                                                              :
    context(ctx),
    textures(nullptr), size(size), pad(pad),
    upToDate{false, false}
{
    const int depth = size.getDepth() + reservedChannels;
    checkChannelNumber(depth);
 
    // decide on packing
    const int maxChannels = 4 * gpu.getLimit(GraphicPipeline::Limit::TEXTURE_IMAGE_UNITS);
    if (depth <= maxChannels) {
        packX = packY = 1;
    }
    else {
        const int channelsPerTexture = ceili(depth, maxChannels);
        for (int i = 1; i*i <= channelsPerTexture; ++i)
            if (channelsPerTexture % i == 0) {
                packX = i;
                packY = channelsPerTexture / i;
            }
    }
}

Storage() [3/3]

Storage::Storage	(	Context &	ctx,
		GraphicPipeline &	gpu,
		const Size	size
	)

Creates a flat storage.

It uses only one texture stacking all the feature maps in a column and has no padding.

Parameters

[in]	ctx	A context
[in]	gpu	A graphic pipeline instance
[in]	size	Storage size

Definition at line 151 of file storage.cpp.

                                                                   :
    context(ctx),
    textures(nullptr), size(size), pad(0),
    upToDate{false, false}
{
    checkChannelNumber(size.getDepth());
    packX = 1;
    packY = size.getDepth() / 4;
}

~Storage()

Storage::~Storage

(

)

Definition at line 162 of file storage.cpp.

                  {
    free();
}

Member Function Documentation

push() [1/3]

void Storage::push ( GraphicPipeline &  gpu, const void *  data  )

private

Definition at line 86 of file storage.cpp.

                                                         {
    const uint8_t* ptr = (const uint8_t*)data;
    const size_t textureSizeBytes =  getTextureWidth() * getTextureHeight() * 4;
 
    const GL::TextureHandler::TextureFormat format = GL::TextureHandler::TextureFormat::RGBAx8;
    glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
    for (int i = 0; i < getNumberOfTextures(); ++i) {
        glBindTexture(GL_TEXTURE_2D, textures[i].handle);
#ifdef BEATMUP_OPENGLVERSION_GLES20
        glTexImage2D(GL_TEXTURE_2D,
            0,
            GL::BITMAP_INTERNALFORMATS[format],
            getTextureWidth(), getTextureHeight(),
            0,
            GL_RGBA,
            GL::BITMAP_PIXELTYPES[format],
            data ? ptr : nullptr);
        if (data)
             ptr += textureSizeBytes;
#else
        glTexStorage2D(GL_TEXTURE_2D, 1, GL::BITMAP_INTERNALFORMATS[format], getTextureWidth(), getTextureHeight());
        if (data) {
            glTexSubImage2D(GL_TEXTURE_2D,
                0, 0, 0, getTextureWidth(), getTextureHeight(),
                GL_RGBA,
                GL::BITMAP_PIXELTYPES[format],
                ptr
            );
            ptr += textureSizeBytes;
        }
#endif
 
        // set dirty flag if no data (the texture needs to be cleared before being used)
        textures[i].dirty = (data == nullptr);
        GL::GLException::check("allocating storage");
    }
 
    upToDate[ProcessingTarget::GPU] = true;
}

allocate() [1/2]

void Storage::allocate

(

GraphicPipeline &

gpu

)

Allocates the storage in GPU memory.

Parameters

[in]

gpu

A graphic pipeline instance

Definition at line 167 of file storage.cpp.

                                           {
    if (textures)
        return;
 
    // setting up textures
    const int count = getNumberOfTextures();
    textures = new Texture[count];
    for (int i = 0; i < count; ++i)
        glGenTextures(1, &textures[i].handle);
 
    push(gpu, nullptr);
}

allocate() [2/2]

void Storage::allocate

(

)

Allocates the storage in RAM.

Definition at line 181 of file storage.cpp.

                       {
    if (memory)
        return;
 
    memory = AlignedMemory(getMemorySize());
    upToDate[ProcessingTarget::CPU] = true;
}

free() [1/2]

void Storage::free

(

GraphicPipeline &

gpu

)

Frees the allocated memory immediately.

Parameters

[in]

gpu

A graphic pipeline instance

Definition at line 190 of file storage.cpp.

                                       {
    // feeing GPU storage
    if (textures != nullptr) {
        for (int i = 0; i < getNumberOfTextures(); ++i)
            glGenTextures(1, &textures[i].handle);
        delete[] textures;
        textures = nullptr;
    }
 
    // freeing CPU storage
    memory.free();
}

free() [2/2]

void Storage::free

(

)

Deferred storage disposal: the textures are put into the GPU recycle bin associated with the context.

Definition at line 204 of file storage.cpp.

                   {
    class Deleter : public GL::RecycleBin::Item {
        GL::handle_t* handles;
        int count;
    public:
        Deleter(const Texture* textures, int count) : handles(new GL::handle_t[count]), count(count) {
            for (int i = 0; i < count; ++i)
                handles[i] = textures[i].handle;
        }
        ~Deleter() {
            glDeleteTextures(count, handles);
            delete[] handles;
        }
    };
 
    // freeing GPU storage
    if (textures != nullptr) {
        context.getGpuRecycleBin()->put(new Deleter(textures, getNumberOfTextures()));
        textures = nullptr;
    }
 
    // freeing CPU storage
    memory.free();
 
    upToDate[ProcessingTarget::CPU] = false;
    upToDate[ProcessingTarget::GPU] = false;
}

pull()

void Storage::pull

(

GraphicPipeline &

gpu

)

Pulls storage data from GPU memory to RAM.

Parameters

[in]

gpu

A graphic pipeline instance

Definition at line 233 of file storage.cpp.

                                       {
    if (!upToDate[ProcessingTarget::GPU])
        throw InconsistentStorageState("No data to pull in the storage");
 
    const int textureSizeBytes = getTextureWidth() * getTextureHeight() * 4;
 
    // allocate / acquire CPU storage
    if (!memory)
        memory = AlignedMemory(textureSizeBytes * getNumberOfTextures());
    uint8_t* ptr = memory.ptr<uint8_t>(0);
 
    glPixelStorei(GL_PACK_ALIGNMENT, 1);
    for (int i = 0; i < getNumberOfTextures(); ++i) {
        gpu.bindOutput(textures[i].handle);
        glReadPixels(0, 0, getTextureWidth(), getTextureHeight(),
             GL_RGBA, GL_UNSIGNED_BYTE,
             ptr
        );
        ptr += textureSizeBytes;
        GL::GLException::check("pulling storage, texture " + std::to_string(i) + " of " + std::to_string(getNumberOfTextures()));
    }
 
    upToDate[ProcessingTarget::CPU] = true;
}

push() [2/3]

void Storage::push

(

GraphicPipeline &

gpu

)

Pushes storage data from RAM to CPU memory.

Parameters

[in]

gpu

A graphic pipeline instance

Definition at line 259 of file storage.cpp.

                                       {
    if (!upToDate[ProcessingTarget::CPU])
        throw InconsistentStorageState("No data to push in the storage");
 
    // allocate on GPU if not yet
    if (!textures) {
        const int count = getNumberOfTextures();
        textures = new Texture[count];
        for (int i = 0; i < count; ++i) {
            glGenTextures(1, &textures[i].handle);
            textures[i].dirty = true;
        }
    }
 
    push(gpu, memory());
}

push() [3/3]

void Storage::push	(	GraphicPipeline &	gpu,
		const float *	hwcData,
		const size_t	numSamples
	)

Pushes a given data to GPU memory.

The data is stored as a 3D array of (height, width, channels) layout (n and n+1 channel values are next to each other in memory). If the array length does not match the storage capacity, an exception is thrown.

Parameters

[in]	gpu	A graphic pipeline instance
[in]	hwcData	The data to push.
[in]	numSamples	The data array length (number of floating point values to push).

Definition at line 277 of file storage.cpp.

                                                                                      {
    RuntimeError::check(numSamples == getSize().volume(), "Data size does not match storage capacity");
 
    // allocate the CPU storage if not yet
    if (!memory) {
        allocate();
        memset(memory(), 0, getMemorySize());
    }
 
#ifdef BEATMUP_DEBUG
    // check the input data range
    for (int i = 0; i < numSamples; ++i)
        OutOfRange::check<float>(hwcData[i], 0, 1, "Data contains a sample falling out of 0..1 range: %0.8f");
#endif
 
    // push
    const int width = size.getWidth(), height = size.getHeight(), depth = size.getDepth();
    const int paddedWidth = width + 2 * pad;
    const int paddedHeight = height + 2 * pad;
    for (int c = 0; c < depth; c+=4)
        for (int y = 0, i = 0; y < height; ++y) {
            color4i* dst = memory.ptr<color4i>(paddedWidth * paddedHeight * c / 4 + paddedWidth * (y + pad) + pad);
            for (int x = 0; x < width; ++x, ++dst, ++i) {
                dst->r = (uint8_t)(hwcData[i * depth + c + 0] * 255);
                dst->g = (uint8_t)(hwcData[i * depth + c + 1] * 255);
                dst->b = (uint8_t)(hwcData[i * depth + c + 2] * 255);
                dst->a = (uint8_t)(hwcData[i * depth + c + 3] * 255);
            }
        }
 
    upToDate[ProcessingTarget::CPU] = true;
    push(gpu);
}

isUpToDate()

bool Beatmup::NNets::Storage::isUpToDate ( ProcessingTarget  target ) const

inline

Checks if the storage is up to date for a given processing target (CPU or GPU).

Parameters

[in]

target

The target (CPU or GPU)

Returns

true if the data is up-to-date.

Definition at line 224 of file storage.h.

{ return upToDate[target]; }

isAllocated()

bool Beatmup::NNets::Storage::isAllocated ( ) const

inline

Returns true if the storage is allocated.

Definition at line 229 of file storage.h.

{ return textures != nullptr || memory; }

getImage()

InternalBitmap * Storage::getImage	(	Context &	ctx,
		GraphicPipeline &	gpu,
		int	channel
	)		const

Converts a feature channel into a bitmap for debugging purposes.

Parameters

[in]	ctx	A context
[in]	gpu	A graphic pipeline instance
[in]	channel	Feature channel number to export

Definition at line 312 of file storage.cpp.

                                                                                       {
    RuntimeError::check(isAllocated(), "The storage contains no data because it is not yet allocated.");
    RuntimeError::check(0 <= channel && channel < size.getDepth(), "Channel number is out of range.");
 
    static const bool crop = true;    // crop the specific channel instead of saving the whole texture
 
    // prepare code
    String code;
    code.printf(BEATMUP_SHADER_CODE(
        uniform sampler2D image;
        varying highp vec2 texCoord;
        void main() {
            lowp float v = texture2D(image, texCoord)[%d];
            gl_FragColor = vec4(v, v, v, 1.0);
        }
    ), channel % 4);
 
    // init bitmap, set as output
    InternalBitmap* bitmap = new InternalBitmap(ctx, PixelFormat::QuadByte,
        crop ? size[0] : getTextureWidth(),
        crop ? size[1] : getTextureHeight(),
        true);
 
    // setup shaders
    GL::FragmentShader shader(gpu, code, GL::Extensions::BEATMUP_DIALECT);
    GL::RenderingProgram program(gpu, shader);
 
    // init binder
    const IntPoint origin = getChannelOrigin(channel / 4 * 4);
    Binder binder(gpu);
    binder.begin(program, *bitmap);
    gpu.setTextureCoordinates(
        crop ?
            IntRectangle(origin.x, origin.y, origin.x + size[0] - 1, origin.y + size[1] - 1) :
            IntRectangle(0, 0, getTextureWidth() - 1, getTextureHeight() - 1),
        IntPoint(getTextureWidth(), getTextureHeight()),
        bitmap->getSize()
    );
 
    // bind input
    bind(0, textures[getChannelTextureNumber(channel / 4 * 4)].handle);
 
    // go
    {
        AbstractBitmap::WriteLock<ProcessingTarget::GPU> lock(*bitmap);
        program.blend();
    }
    gpu.pullPixels(*bitmap);
 
    return bitmap;
}

getNumberOfTextures()

int Storage::getNumberOfTextures

(

)

const

Returns total number of textures in the storage.

Definition at line 365 of file storage.cpp.

                                       {
    return ceili(size.getDepth(), 4 * packX * packY);
}

getChannelTextureNumber()

int Storage::getChannelTextureNumber

(

int

channel

)

const

Returns number of texture containing a given channel.

Definition at line 370 of file storage.cpp.

                                                      {
#ifdef BEATMUP_DEBUG
    checkChannelNumber(channel);
    OutOfRange::check(channel, 0, size[2] - 1, "Channel out of range: %d");
#endif
    return channel / 4 / (packX * packY);
}

getChannelOrigin()

IntPoint Storage::getChannelOrigin

(

int

channel

)

const

Returns origin in pixels of a given channel within the texture containing it.

Definition at line 379 of file storage.cpp.

                                                    {
#ifdef BEATMUP_DEBUG
    checkChannelNumber(channel);
#endif
    return IntPoint(
        pad + (channel / 4 % packX) * (pad + size[0]),
        pad + ((channel / 4 / packX) % packY) * (pad + size[1])
    );
}

getTextureWidth()

int Storage::getTextureWidth

(

)

const

Returns width in pixels of all the textures.

Definition at line 390 of file storage.cpp.

                                   {
    return (size.getWidth() + pad) * packX + pad;
}

getTextureHeight()

int Storage::getTextureHeight

(

)

const

Returns height in pixels of all the textures.

Definition at line 395 of file storage.cpp.

                                    {
    return (size.getHeight() + pad) * packY + pad;
}

getPadding()

int Beatmup::NNets::Storage::getPadding ( ) const

inline

Returns storage padding.

Padding is zero-valued pixels on each side of the storage.

Definition at line 268 of file storage.h.

{ return pad; }

getSize()

Size Beatmup::NNets::Storage::getSize ( ) const

inline

Returns storage size in pixels.

Definition at line 273 of file storage.h.

{ return size; }

getMemorySize()

size_t Beatmup::NNets::Storage::getMemorySize ( ) const

inline

Definition at line 279 of file storage.h.

{ return (size_t)getTextureWidth() * getTextureHeight() * getNumberOfTextures() * 4; }

getMemory() [1/2]

AlignedMemory& Beatmup::NNets::Storage::getMemory ( )

inline

Returns the storage memory, if allocated in RAM (for CPU).

Definition at line 284 of file storage.h.

{ return memory; }

getMemory() [2/2]

const AlignedMemory& Beatmup::NNets::Storage::getMemory ( ) const

inline

Definition at line 285 of file storage.h.

{ return memory; }

checkChannelNumber()

static void Beatmup::NNets::Storage::checkChannelNumber ( int  channel )

inlinestatic

Checks whether a channel number points to the first channel in a texture.

Throws an exception otherwise.

Definition at line 290 of file storage.h.

                                                               {
                class InvalidChannelNumber : public Exception {
                public:
                    InvalidChannelNumber(int channel): Exception("Invalid channel number / number of channels: %d", channel) {}
                };
                if (channel % 4 != 0)
                    throw InvalidChannelNumber(channel);
            }

Friends And Related Function Documentation

Binder

friend class Binder

friend

Definition at line 134 of file storage.h.

View

friend class View

friend

Definition at line 135 of file storage.h.

Member Data Documentation

context

Context& Beatmup::NNets::Storage::context

private

Definition at line 143 of file storage.h.

textures

Texture* Beatmup::NNets::Storage::textures

private

Definition at line 144 of file storage.h.

memory

AlignedMemory Beatmup::NNets::Storage::memory

private

data storage in RAM

Definition at line 145 of file storage.h.

size

const Size Beatmup::NNets::Storage::size

private

Definition at line 146 of file storage.h.

pad

const int Beatmup::NNets::Storage::pad

private

padding in pixels added along width and height dimensions

Definition at line 147 of file storage.h.

packX

int Beatmup::NNets::Storage::packX

private

Definition at line 148 of file storage.h.

packY

int Beatmup::NNets::Storage::packY

private

number of blocks of 4 channels per texture (spatial packing)

Definition at line 148 of file storage.h.

upToDate

bool Beatmup::NNets::Storage::upToDate[2]

private

Definition at line 149 of file storage.h.

---

The documentation for this class was generated from the following files:

• core/nnets/storage.h
• core/nnets/storage.cpp