Is there a way to access the current camera frame in Android?

Question

I wanna capture the frames do same image processing. How can I do that using the EasyAR Android SDK?

I try to get the frame from the stream, like that:

Frame frame = streamer.peek();
if (frame.images().size()>0) {
    int size = frame.images().get(0).buffer().size();
    if (size>0) {
        byte[] bytes = new byte[size];
        frame.images().get(0).buffer().copyTo(bytes, size);
    }
}

But it is not working.​

Answer 1

The first problem is that the second parameter of the method copyTo(), is supposed to be zero. I found this information in the Buffer Class Documentation:

https://www.easyar.com/doc/EasyAR%20SDK/API%20Reference/2.0/Buffer.html

The second problem is that the buffer byte is on the YUV_NV21 image format. You can verify that using: 

    Frame frame = streamer.peek();
    Log.i(TAG, "FORMAT: " + frame.images().get(0).format() );

format() returns an int, where 2 means YUV_NV21 image format. You can verify all the formats on the Image Class Documentation:

https://www.easyar.com/doc/EasyAR%20SDK/API%20Reference/2.0/Image.html

To convert the YUV_NV21 to an RGB image I used my solution that is explained on StackOverflow:

https://stackoverflow.com/a/12702836/1178478

So the code that converts the Frame from the EasyAR streamer to a Bitmap is this:

Bitmap getBitmapFromFrameStreamer(cn.easyar.Frame frame) {
    Bitmap frame_bm = null;

    if (frame.images().size()>0) {
        int size   = frame.images().get(0).buffer().size();
        int width  = frame.images().get(0).width();
        int height = frame.images().get(0).height();
        int format = frame.images().get(0).format();
        if (size>0) {
            byte[] bytes = new byte[size];
            frame.images().get(0).buffer().copyTo(bytes, 0);

            // 2 == YUV_NV21
            if (format==2) {
                int[] pixels = convertYUV420_NV21toRGB8888(bytes, width, height);
                frame_bm = Bitmap.createBitmap(pixels, width, height, Bitmap.Config.ARGB_8888);
                if (frame_bm!=null)
                    Log.i(TAG, "frame bm: " + frame_bm.getWidth() + "x" + frame_bm.getHeight());
            }
        }
    }

    return frame_bm;
}

To use this method:

Frame frame = streamer.peek();
Bitmap frame_bm = getBitmapFromStreamer(frame);

The method convertYUV420_NV21toRGB8888() can be found on the StackOverflow linked before.

This code was added to the HelloAR.java from the HelloARVideo Android sample.