Wednesday, May 4, 2016

JavaFX and webcams

There is an excellent webcam library available for Java. Unfortunately, when trying to integrate it with a JavaFX application I got into some kind of threading trouble. I was able to escape this by creating my own thread to deal with the webcam.

My demo is available on github as an Eclipse project. You'll need to download the aforementioned library and set up its three JAR files in your build path to make it work. Beyond that, it should be straightforward.

Here are some highlights from the code I wrote.

The ImageThread class handles interactions with the webcam. It requires a function to actually perform the rendering on the interface, but aside from that it takes care of dealing with the webcam. I separated out the rendering function so that anyone can just borrow this class wholesale for whatever application they are writing. Note that it is pretty easy to add additional image processing to the threaded loop.

The implementation is straightforward. When the thread starts, it opens the webcam and begins counting frames and time. Whenever the webcam has a new image available, it grabs the image and calls the renderer with both the image and the current frame rate.

Be sure that your rendering function invokes Platform.runLater() to avoid threading problems!

package edu.hendrix.webcamdemo;

import java.awt.image.BufferedImage;
import java.util.function.BiConsumer;

import com.github.sarxos.webcam.Webcam;

public class ImageThread extends Thread {
  private boolean quit = false;
  private int frames;
  private BiConsumer<Bufferedimage,Double> renderer;
 
  public ImageThread(BiConsumer<Bufferedimage,Double> renderer) {
    this.renderer = renderer;
  }
 
  public void quit() {quit = true;}
 
  @Override
  public void run() {
    Webcam webcam = Webcam.getDefault();
    webcam.open();
    frames = 0;
    long start = System.currentTimeMillis();
    while (!quit) {
      if (webcam.isImageNew()) {
        BufferedImage img = webcam.getImage();
        frames += 1;
        renderer.accept(img, 1000.0*frames / (System.currentTimeMillis() - start));
      }
    }
    webcam.close();
  }
}

The WebcamDemoController class mediates between the GUI and the ImageThread. This gives a nice example as to how to provide a rendering function for the GUI.

package edu.hendrix.webcamdemo;

import javafx.application.Platform;
import javafx.fxml.FXML;
import javafx.scene.canvas.Canvas;
import javafx.scene.canvas.GraphicsContext;
import javafx.scene.control.TextField;

import java.awt.image.BufferedImage;

public class WebcamDemoController implements Quittable {
  @FXML Canvas image;
 
  @FXML TextField frameRate;
 
  ImageThread renderer;
 
  int frames;
 
  public static void render(BufferedImage img, Canvas canv) {
    double cellWidth = canv.getWidth() / img.getWidth();
    double cellHeight = canv.getHeight() / img.getHeight();
    GraphicsContext g = canv.getGraphicsContext2D();
    for (int x = 0; x < img.getWidth(); x++) {
      for (int y = 0; y < img.getHeight(); y++) {
        g.setFill(ColorChannel.buildColorFrom(img.getRGB(x, y)));
        g.fillRect(x * cellWidth, y * cellHeight, cellWidth, cellHeight);
      }
    }
  }
 
  @FXML
  void initialize() {
    renderer = new ImageThread((img, rate) -> Platform.runLater(() -> {
      render(img, image);
      frameRate.setText(String.format("%4.2f", rate));
    }));
    renderer.start();
    frameRate.setEditable(false);
  }

  @Override
  public void quit() {
    renderer.quit();
  }
}

Finally, I also created an Enum to handle the ARGB conversions. Since I'm interested in image processing, it is helpful to be able to directly access the pertinent color channels.

package edu.hendrix.webcamdemo;

import java.util.EnumMap;

import javafx.scene.paint.Color;

public enum ColorChannel {
  ALPHA {
    @Override
    public int shift() {
      return 24;
    }
  }, RED {
    @Override
    public int shift() {
      return 16;
    }
  }, GREEN {
    @Override
    public int shift() {
      return 8;
    }
  }, BLUE {
    @Override
    public int shift() {
      return 0;
    }
  };
 
  abstract public int shift();
 
  public int extractFrom(int pixel) {
    return (pixel >> shift()) & 0xFF;
  }
 
  public static int buildPixelFrom(EnumMap colors) {
    int pixel = 0;
    for (ColorChannel c: values()) {
      pixel += (colors.containsKey(c) ? colors.get(c) : 0) << c.shift();
    }
    return pixel;
  }
 
  public static javafx.scene.paint.Color buildColorFrom(int argb) {
    return Color.rgb(ColorChannel.RED.extractFrom(argb), 
        ColorChannel.GREEN.extractFrom(argb), 
        ColorChannel.BLUE.extractFrom(argb), 
        ColorChannel.ALPHA.extractFrom(argb) / 255.0);
  }
 
  public static EnumMap<ColorChannel,Integer> buildChannelsFrom(int argb) {
    EnumMap<ColorChannel,Integer> channels = new EnumMap<>(ColorChannel.class);
    for (ColorChannel c: values()) {
      channels.put(c, c.extractFrom(argb));
    }
    return channels;
  }
}

Monday, April 18, 2016

Overflow and comparators

When writing a Comparator for integer-valued types in Java, the following implementation seems straightforward and clever:

private Comparator<Integer> c = (i, j) -> i - j;

When I used this implementation when writing unit tests for a Data Structures assignment (implementing a heap), much to my surprise the results were rather buggy. I was generating random integers and throwing them in the heap, followed by checking the heap property. When I replaced this implementation with the following alternative, everything worked perfectly:

private Comparator<Integer> c = (i, j) -> i < j ? -1 : i > j ? 1 : 0;

So what's the difference? Overflow. Imagine i = 2135909750 and j = -21314406. The subtraction yields -2137743140, which, being negative, is not exactly the desired result. Of course, with roles reversed underflow is also a hazard.

The lesson for me today is that sometimes, when trying to be clever and succinct, don't be too clever!

Saturday, February 6, 2016

Open-Source Software and Societal Infrastructure

I read two political news articles today that both suggest the importance of software freedom for the sake of our societal infrastructure.  The first article describes the Repair Coalition, a political lobbying group that seeks to make it both legal and possible for any repair person to repair any device.  In the context of repairs, the freedom to redistribute modified copies of software seems to me to have particular importance.  There is no guarantee that the manufacturer will release a defect-free product, and giving everyone the opportunity to improve upon it has great potential for benefiting all consumers.

The second article describes a controversy around the vote-tallying machines employed in the 2016 Iowa Caucus. These machines were manufactured and programmed by Microsoft.  The company is accused of a conflict-of-interest, as many of their employees have donated to campaigns for active candidates in the current campaign.  It is in our collective interest that these machines be trustworthy.  The only real path forward is to allow their software to be available for audits to any interested person, with no special permission needed.

In both cases, what we have is instances of closing software with the result of social exclusion.  Those who wish to repair devices are excluded from doing so by manufacturers who refuse to make the source code available.  Those who wish to audit the result of electronic vote-tallies are excluded from doing so by the manufacturer.  And cases like this are just the tip of the iceberg: for example, what about the passengers in self-driving cars?

For what positive end do we have this exclusion?  Must intellectual property be concealed to make it economically feasible for manufacturers to create their products?  Not all manufacturers agree. The pervasiveness of software throughout our society seems to me to correlate to an increase in the force of arguments for software freedom.


Sunday, September 20, 2015

Displaying images from files in JavaFX

As I continue to both employ JavaFX in my own code and teach JavaFX in my courses, I am slowly learning how to perform tasks I used to do in Swing in new ways.  The latest example is the all-important task of opening image files.

The strategy employed in this example is to set the image file as a background for a JavaFX Pane object.  Opening the image is a straightforward use of the JavaFX file chooser dialog.  Error handling is done via a simple error dialog created as an Alert.  Displaying the image requires the following steps:

  • Transform the File object into a URL, and use this to create an Image object.
  • Create a BackgroundSize for the object.  
    • By setting the height and width to AUTO, the image will automatically resize as the window displaying it is resized. 
    • It will not deform the image; if the aspect ratio of the displaying window does not match the image, the image will be truncated.
  • Create a BackgroundImage object.
    • The first parameter passes the Image object to use as its basis.
    • The second and third parameters control whether the object is repeated in the X and Y directions.  For this example, I deemed this undesirable.
    • The fourth parameter centers the image in the background.
    • The final parameter is the BackgroundSize object created before. 
  • Finally, we designate the BackgroundImage as the Pane's background.

import java.io.File;
import java.net.MalformedURLException;

import javafx.fxml.FXML;
import javafx.scene.control.Alert;
import javafx.scene.control.Alert.AlertType;
import javafx.scene.control.Button;
import javafx.scene.image.Image;
import javafx.scene.layout.Background;
import javafx.scene.layout.BackgroundImage;
import javafx.scene.layout.BackgroundPosition;
import javafx.scene.layout.BackgroundRepeat;
import javafx.scene.layout.BackgroundSize;
import javafx.scene.layout.Pane;
import javafx.stage.FileChooser;
import javafx.stage.FileChooser.ExtensionFilter;

public class DemoController {
 @FXML
 Pane pane;
 
 @FXML
 Button open;
 
 @FXML
 void initialize() {
  open.setOnAction(event -> imageOpener());
 }
 
 void imageOpener() {
  File imgFile = openImage();
  try {
   if (imgFile != null) {
    displayImage(imgFile);
   } 
  } catch (MalformedURLException e) {
   showError(e);
  }
 }
 
 File openImage() {
  FileChooser fileChooser = new FileChooser();
  fileChooser.setTitle("Select an Image File");
  fileChooser.getExtensionFilters().add(new ExtensionFilter(
    "Image Files", "*.png", "*.jpg", "*.gif"));
  return fileChooser.showOpenDialog(null);
 }
 
 void displayImage(File imgFile) throws MalformedURLException {
  Image background = new Image(imgFile.toURI().toURL().toString());
  BackgroundSize size = new BackgroundSize(BackgroundSize.AUTO,
    BackgroundSize.AUTO, false, false, true, true);
  BackgroundImage bimg = new BackgroundImage(background,
    BackgroundRepeat.NO_REPEAT, BackgroundRepeat.NO_REPEAT,
    BackgroundPosition.CENTER, size);
  pane.setBackground(new Background(bimg));
 }
 
 void showError(Exception e) {
  Alert info = new Alert(AlertType.ERROR);
  info.setTitle("Image not found");
  info.setHeaderText("Image not found");
  info.setContentText(e.getMessage());
  info.showAndWait();
 }
}

Dragging objects with a mouse in JavaFX

Dragging objects is pretty straightforward in JavaFX as long as some key concepts are kept straight.  If we go with the strategy of attaching the event handler to the object to be dragged, then the very first time that the onMouseDragged event is invoked, the (x,y) coordinates given by the event parameter will be the location on the object that was clicked.

If we then move the object, on subsequent invocations of the event as a result of moving the mouse, the (x,y) coordinates will be the relative distance between the mouse position at the time of the preceding event and the current mouse position.  This works because the new position of the object is determined by the previous mouse location.

In either case, it works pretty well to use the (x,y) coordinate of the mouse as an offset by which to translate the object.  The code below demonstrates the concept nicely, provided that a suitable GUI has been created using SceneBuilder and a suitable main() has also been created.  The drag() method is set up to work properly with a draggable Node of any subtype.

import javafx.fxml.FXML;
import javafx.scene.Node;
import javafx.scene.input.MouseEvent;
import javafx.scene.layout.Pane;
import javafx.scene.shape.Circle;

public class DemoController {
 @FXML
 Pane pane;
 
 @FXML
 Circle ball1, ball2;
 
 @FXML
 void initialize() {
  ball1.setOnMouseDragged(event -> drag(event));
  ball2.setOnMouseDragged(event -> drag(event));
 }
 
 public void drag(MouseEvent event) {
  Node n = (Node)event.getSource();
  n.setTranslateX(n.getTranslateX() + event.getX());
  n.setTranslateY(n.getTranslateY() + event.getY());
 }
}

Tuesday, September 8, 2015

Be careful with operator precedence

I recently encountered the following bug while programming in Java. I've taken it out of context and created the program below for illustrative purposes.

The goal is to add two byte values while avoiding overflow. To this end, we cast them to integers, and we use bit-masking to avoid sign extension when casting.  The unsuccessful version is highlighted in red, while the successful version is green:

public class ConvertBytes {
 public static void main(String[] args) {
  byte a = 127;
  byte b = 126;
  int bad = ((int)a) & 0xff + ((int)b) & 0xff;
  System.out.println(bad);
  int good = ((int)a & 0xff) + ((int)b & 0xff);
  System.out.println(good);
 }
}

When this program is executed, bad is 125 and good is 253.

I can't believe that after 13 years as an active Java programmer (with 11 years before that as an active C++ programmer) I made so elementary a mistake, but I'm posting it here in hopes it will be a useful warning to others.  For whatever reason, I assumed that bitwise-and had a higher precedence than addition, perhaps because it is a traditionally "multiplicative" operator.  As it turns out, its precedence is actually much lower.

A further observation is that this is not what I thought the bug was originally.  At first, I thought I had somehow messed up the casting.  It was only after I rechecked it carefully that I realized I was killing the higher-order bits with the bitwise and.