Implementing a multi-touch event system to deliver blob events to registered widgets (and creating a demo photo application with inertia) part 2 of 2

In the previous post we discussed the event queue and the abstract base class for the widgets. Now we will concentrate on creating some widgets that we can use by extending the base class, and we will look at setting up the queue, registering widgets, and calling the processEvents() method in our programs main loop. By the end of this post we should be able to implement the photo application shown in the image below.

The photo application we will implement by the end of this post.

The first widget we will declare is the cRectangle object. The purpose of this widget will be to render a quad with a specific color, but it will also allow us to define the method for evaluating containment, containsPoint(), and the update() method for processing the events.

class cRectangle : public cWidget {
  private:
    
  protected:
	point dimensions;
    
  public:
	cRectangle(point translate, double rotate, double scale, point dimensions, rgba color);
	~cRectangle();
	
	virtual bool containsPoint(const point& p) const;
	virtual void update();
	virtual void render();
};

In the containsPoint() method we are essentially applying a transformation to the given point to find its representation in the local coordinate system of the rectangle. This transformation is a composition of the scaling, translation, and rotation transformations.

In we have scaling:
\begin{align}
x'&=s_xx\\
y'&=s_yy
\end{align}
Translation:
\begin{align}
x'&=x+t_x\\
y'&=y+t_y
\end{align}
Rotation:
\begin{align}
x'&=x\cos{\theta}-y\sin{\theta}\\
y'&=x\sin{\theta}+y\cos{\theta}
\end{align}

More information including transformations in using homogeneous coordinates can be found in this document. Once we have the given point's representation in the rectangle's local coordinate system we do a simple check to see if it's contained within the box (axis-aligned) defined by the rectangle's dimensions.

In the update() method we are only concerned with processing an events vector of size 0, 1, or 2. In the last post we prioritized our blob events by their identification (older events have a lower id). This way when our widget receives more than two events we still process the original two and discard the newer events. We can break this method down into a set of cases:

Events vector has size zero or vector is size one with BLOB_UP event:

  • apply inertia

Events vector has size one

  • BLOB_DOWN event - bring widget to surface
  • BLOB_MOVE event - translate widget

Events vector has size two

  • BLOB_DOWN and BLOB_DOWN - bring widget to surface
  • BLOB_DOWN and BLOB_MOVE - bring widget to surface and translate widget
  • BLOB_DOWN and BLOB_UP - bring widget to surface
  • BLOB_MOVE and BLOB_MOVE - translate, rotate, and scale widget
  • BLOB_MOVE and BLOB_UP - translate widget

Lastly, the render() method simply renders the quad after applying the widget's transformation properties.

cRectangle::cRectangle(point translate, double rotate, double scale, point dimensions, rgba color) : cWidget(translate, rotate, scale, color), dimensions(dimensions) {
//	type = RECTANGLE;
}

cRectangle::~cRectangle() { }

bool cRectangle::containsPoint(const point& p) const {
	point p0(p.x - translate.x, p.y - translate.y);
	point p1(p0.x * cos(-rotate) + p0.y * -sin(-rotate), p0.x * sin(-rotate) + p0.y * cos(-rotate));
	return (p1.x >= -dimensions.x / 2 * scale && p1.x <= dimensions.x / 2 * scale && p1.y >= -dimensions.y / 2 * scale && p1.y <= dimensions.y / 2 * scale);
}

void cRectangle::update() {
	if (events.size() == 0 || events.size() == 1 && events[0]->event == BLOB_UP) {
		translate += translate_previous; translate_previous *= 0.9;
		rotate    += rotate_previous;    rotate_previous    *= 0.9;
	} else if (events.size() == 1) {
		if (events[0]->event == BLOB_DOWN) bringForward();
		if (events[0]->event == BLOB_MOVE) translateWidget(events[0]->location-events[0]->origin);
	} else if (events.size() > 1) {
		if (events[0]->event == BLOB_DOWN || events[1]->event == BLOB_DOWN) {
			bringForward();
			if (events[0]->event == BLOB_MOVE) {
				translateWidget(events[0]->location - events[0]->origin);
			} else if (events[1]->event == BLOB_MOVE) {
				translateWidget(events[1]->location - events[1]->origin);
			}
		} else if (events[0]->event == BLOB_MOVE && events[1]->event == BLOB_MOVE) {
			point p0 = events[1]->origin + events[0]->origin;
			point p1 = events[1]->location + events[0]->location;
			point p2 = (p1 - p0) * 0.5;
			translateWidget(p2);
			rotateWidget(events[1]->origin - events[0]->origin, events[1]->location - events[0]->location, p1/2);
			scaleWidget(events[1]->origin - events[0]->origin, events[1]->location - events[0]->location, p1/2);
		} else if (events[0]->event == BLOB_MOVE && events[1]->event == BLOB_UP) {
			translateWidget(events[0]->location-events[0]->origin);
		} else if (events[1]->event == BLOB_MOVE && events[0]->event == BLOB_UP) {
			translateWidget(events[1]->location-events[1]->origin);
		}
	}
}

void cRectangle::render() {
	glDisable(GL_TEXTURE_2D);
	glColor4f(color.r, color.g, color.b, color.a);
	glPushMatrix();
	glTranslatef(translate.x, translate.y, 0);
	glRotatef(rotate * 180 / M_PI, 0, 0, 1);
	glScalef(scale, scale, scale);
	glBegin(GL_QUADS);
	glVertex3f(-dimensions.x / 2, -dimensions.y / 2, 0.0);
	glVertex3f(-dimensions.x / 2,  dimensions.y / 2, 0.0);
	glVertex3f( dimensions.x / 2,  dimensions.y / 2, 0.0);
	glVertex3f( dimensions.x / 2, -dimensions.y / 2, 0.0);
	glEnd();
	glPopMatrix();
};

We will now declare a cTexture widget to extend cRectangle. This widget will use the containsPoint() and update() methods of its base class, but we will override the render() method to render a texture.

class cTexture : public cRectangle {
  private:
    
  protected:
	GLuint texture;

  public:
	cTexture(std::string filename);
	cTexture(std::string filename, point translate, double rotate, double scale);
	~cTexture();

	virtual void render();
};

The definition of cTexture is straightforward.

cTexture::cTexture(std::string filename) : cRectangle(point(0.0, 0.0), 0.0, 1.0, point(1.0, 1.0), rgba(1.0, 1.0, 1.0, 1.0)) {
	SDL_Surface *s = IMG_Load(filename.c_str());
	dimensions.x = s->w; dimensions.y = s->h;
	setupTexture(texture, s);
	SDL_FreeSurface(s);
}

cTexture::cTexture(std::string filename, point translate, double rotate, double scale) : cRectangle(translate, rotate, scale, point(1.0, 1.0), rgba(1.0, 1.0, 1.0, 1.0)) {
	SDL_Surface *s = IMG_Load(filename.c_str());
	dimensions.x = s->w; dimensions.y = s->h;
	setupTexture(texture, s);
	SDL_FreeSurface(s);
}

cTexture::~cTexture() {
	deleteTexture(texture);
}

void cTexture::render() {
	glEnable(GL_TEXTURE_2D);
	glBindTexture(GL_TEXTURE_2D, texture);
	glColor4f(color.r, color.g, color.b, color.a);
	glPushMatrix();
	glTranslatef(translate.x, translate.y, 0);
	glRotatef(rotate * 180 / M_PI, 0, 0, 1);
	glScalef(scale, scale, scale);
	glBegin(GL_QUADS);
	glTexCoord2f(0.0, 0.0); glVertex3f(-dimensions.x / 2, -dimensions.y / 2, 0.0);
	glTexCoord2f(0.0, 1.0); glVertex3f(-dimensions.x / 2,  dimensions.y / 2, 0.0);
	glTexCoord2f(1.0, 1.0); glVertex3f( dimensions.x / 2,  dimensions.y / 2, 0.0);
	glTexCoord2f(1.0, 0.0); glVertex3f( dimensions.x / 2, -dimensions.y / 2, 0.0);
	glEnd();
	glPopMatrix();
}

At this point we have what we need to create our photo application, but we will add one more widget to add a border to our texture, cTextureBorder. This object requires two textures: the photo and the corners texture (for rendering rounded corners).

class cTextureBorder : public cTexture {
  private:
    
  protected:
	GLuint corners;

  public:
	cTextureBorder(std::string filename, std::string corners);
	cTextureBorder(std::string filename, std::string corners, point translate, double rotate, double scale);
	~cTextureBorder();

	void renderBorders();
	virtual void render();
};

The implementation of this widget could use some work. The border widths are hardcoded, and the photo's ratio isn't preserved after applying the borders. With photos at HD resolutions these issues aren't too severe, but we'll correct them eventually.

cTextureBorder::cTextureBorder(std::string filename, std::string corners) : cTexture(filename) {
	SDL_Surface *s = IMG_Load(corners.c_str());
	setupTexture(this->corners, s);
	SDL_FreeSurface(s);
}

cTextureBorder::cTextureBorder(std::string filename, std::string corners, point translate, double rotate, double scale) :
	cTexture(filename, translate, rotate, scale) {
	SDL_Surface *s = IMG_Load(corners.c_str());
	setupTexture(this->corners, s);
	SDL_FreeSurface(s);
}

cTextureBorder::~cTextureBorder() {
	deleteTexture(corners);
}

void cTextureBorder::renderBorders() {
	glDisable(GL_TEXTURE_2D);
	glColor4f(color.r, color.g, color.b, color.a * 0.7);
	glBegin(GL_QUADS);
	glVertex3f(-dimensions.x / 2, -dimensions.y / 2 + 36, 0.0);
	glVertex3f(-dimensions.x / 2, dimensions.y / 2 - 36, 0.0);
	glVertex3f(-dimensions.x / 2 + 36, dimensions.y / 2 - 36, 0.0);
	glVertex3f(-dimensions.x / 2 + 36, -dimensions.y / 2 + 36, 0.0);
	glVertex3f(-dimensions.x / 2 + 36, dimensions.y / 2 - 36, 0.0);
	glVertex3f(-dimensions.x / 2 + 36, dimensions.y / 2, 0.0);
	glVertex3f( dimensions.x / 2 - 36, dimensions.y / 2, 0.0);
	glVertex3f( dimensions.x / 2 - 36, dimensions.y / 2 - 36, 0.0);
	glVertex3f(dimensions.x / 2 - 36, -dimensions.y / 2 + 36, 0.0);
	glVertex3f(dimensions.x / 2 - 36, dimensions.y / 2 - 36, 0.0);
	glVertex3f(dimensions.x / 2, dimensions.y / 2 - 36, 0.0);
	glVertex3f(dimensions.x / 2, -dimensions.y / 2 + 36, 0.0);
	glVertex3f(-dimensions.x / 2 + 36, -dimensions.y / 2, 0.0);
	glVertex3f(-dimensions.x / 2 + 36, -dimensions.y / 2 + 36, 0.0);
	glVertex3f( dimensions.x / 2 - 36, -dimensions.y / 2 + 36, 0.0);
	glVertex3f( dimensions.x / 2 - 36, -dimensions.y / 2, 0.0);
	glEnd();


	glEnable(GL_TEXTURE_2D);
	glBindTexture(GL_TEXTURE_2D, corners);
	glBegin(GL_QUADS);
	glTexCoord2f(0.0, 0.0); glVertex3f(-dimensions.x / 2, -dimensions.y / 2, 0.0);
	glTexCoord2f(0.0, 0.5); glVertex3f(-dimensions.x / 2, -dimensions.y / 2 + 36, 0.0);
	glTexCoord2f(0.5, 0.5); glVertex3f(-dimensions.x / 2 + 36, -dimensions.y / 2 + 36, 0.0);
	glTexCoord2f(0.5, 0.0); glVertex3f(-dimensions.x / 2 + 36, -dimensions.y / 2, 0.0);
	glTexCoord2f(0.0, 0.5); glVertex3f(-dimensions.x / 2, dimensions.y / 2 - 36, 0.0);
	glTexCoord2f(0.0, 1.0); glVertex3f(-dimensions.x / 2, dimensions.y / 2, 0.0);
	glTexCoord2f(0.5, 1.0); glVertex3f(-dimensions.x / 2 + 36, dimensions.y / 2, 0.0);
	glTexCoord2f(0.5, 0.5); glVertex3f(-dimensions.x / 2 + 36, dimensions.y / 2 - 36, 0.0);
	glTexCoord2f(0.5, 0.5); glVertex3f(dimensions.x / 2 - 36, dimensions.y / 2 - 36, 0.0);
	glTexCoord2f(0.5, 1.0); glVertex3f(dimensions.x / 2 - 36, dimensions.y / 2, 0.0);
	glTexCoord2f(1.0, 1.0); glVertex3f(dimensions.x / 2, dimensions.y / 2, 0.0);
	glTexCoord2f(1.0, 0.5); glVertex3f(dimensions.x / 2, dimensions.y / 2 - 36, 0.0);
	glTexCoord2f(0.5, 0.0); glVertex3f(dimensions.x / 2 - 36, -dimensions.y / 2, 0.0);
	glTexCoord2f(0.5, 0.5); glVertex3f(dimensions.x / 2 - 36, -dimensions.y / 2 + 36, 0.0);
	glTexCoord2f(1.0, 0.5); glVertex3f(dimensions.x / 2, -dimensions.y / 2 + 36, 0.0);
	glTexCoord2f(1.0, 0.0); glVertex3f(dimensions.x / 2, -dimensions.y / 2, 0.0);
	glEnd();
}

void cTextureBorder::render() {
	glPushMatrix();
	glTranslatef(translate.x, translate.y, 0);
	glRotatef(rotate * 180 / M_PI, 0, 0, 1);
	glScalef(scale, scale, scale);

	renderBorders();

	glColor4f(color.r, color.g, color.b, color.a);
	glBindTexture(GL_TEXTURE_2D, texture);
	glBegin(GL_QUADS);
	glTexCoord2f(0.0, 0.0); glVertex3f(-dimensions.x / 2 + 36, -dimensions.y / 2 + 36, 0.0);
	glTexCoord2f(0.0, 1.0); glVertex3f(-dimensions.x / 2 + 36,  dimensions.y / 2 - 36, 0.0);
	glTexCoord2f(1.0, 1.0); glVertex3f( dimensions.x / 2 - 36,  dimensions.y / 2 - 36, 0.0);
	glTexCoord2f(1.0, 0.0); glVertex3f( dimensions.x / 2 - 36, -dimensions.y / 2 + 36, 0.0);
	glEnd();

	glPopMatrix();
}

In the initialization part of our program we create our widget instances, create our event queue instance, and register our widgets with the event queue.

	cTextureBorder t0("media/nature/img0.jpg", "media/corners.png", point(96,54), 0, 0.1);
	cTextureBorder t1("media/nature/img1.jpg", "media/corners.png", point(288,54), 0, 0.1);
	cTextureBorder t2("media/nature/img2.jpg", "media/corners.png", point(480,54), 0, 0.1);
	cTextureBorder t3("media/nature/img3.jpg", "media/corners.png", point(672,54), 0, 0.1);
	cTextureBorder t4("media/nature/img4.jpg", "media/corners.png", point(96,162), 0, 0.1);
	cTextureBorder t5("media/nature/img5.jpg", "media/corners.png", point(288,162), 0, 0.1);
	cTextureBorder t6("media/nature/img6.jpg", "media/corners.png", point(480,162), 0, 0.1);
	cTextureBorder t7("media/nature/img7.jpg", "media/corners.png", point(672,162), 0, 0.1);
	cTextureBorder t8("media/nature/img8.jpg", "media/corners.png", point(96,270), 0, 0.1);
	cTextureBorder t9("media/nature/img9.jpg", "media/corners.png", point(288,270), 0, 0.1);
	cTextureBorder t10("media/nature/img10.jpg", "media/corners.png", point(480,270), 0, 0.1);
	cTextureBorder t11("media/nature/img11.jpg", "media/corners.png", point(672,270), 0, 0.1);
	
	cQueue queue;

	queue.registerWidget(t0);	queue.registerWidget(t1);	queue.registerWidget(t2);	queue.registerWidget(t3);
	queue.registerWidget(t4);	queue.registerWidget(t5);	queue.registerWidget(t6);	queue.registerWidget(t7);
	queue.registerWidget(t8);	queue.registerWidget(t9);	queue.registerWidget(t10);	queue.registerWidget(t11);

Finally, in our programs main loop, we call the processEvents() method after calling one of our trackers' trackBlobs() methods.

		queue.processEvents(cvblobslib ? tracker.getBlobs() : tracker2.getBlobs());

In my next post we'll discuss a calibration method for multi-touch surfaces based on barycentric coordinates.

Download this project: event.tar.bz2

Leave a Reply

Your email address will not be published. Required fields are marked *