Is there a way to access the current camera frame in Android?

0 votes
asked Jun 6, 2018 by derzuomaia (1,190 points)

I wanna capture the frames do same image processing. How can I do that using the EasyAR Android SDK?

I try to get the frame from the stream, like that:

Frame frame = streamer.peek();
if (frame.images().size()>0) {
    int size = frame.images().get(0).buffer().size();
    if (size>0) {
        byte[] bytes = new byte[size];
        frame.images().get(0).buffer().copyTo(bytes, size);
    }
}

But it is not working.‚Äč

1 Answer

+1 vote
answered Jun 6, 2018 by derzuomaia (1,190 points)
edited Jun 6, 2018 by derzuomaia
 
Best answer

The first problem is that the second parameter of the method copyTo(), is supposed to be zero. I found this information in the Buffer Class Documentation:

https://www.easyar.com/doc/EasyAR%20SDK/API%20Reference/2.0/Buffer.html

The second problem is that the buffer byte is on the YUV_NV21 image format. You can verify that using: 

    Frame frame = streamer.peek();
    Log.i(TAG, "FORMAT: " + frame.images().get(0).format() );

format() returns an int, where 2 means YUV_NV21 image format. You can verify all the formats on the Image Class Documentation:

https://www.easyar.com/doc/EasyAR%20SDK/API%20Reference/2.0/Image.html

To convert the YUV_NV21 to an RGB image I used my solution that is explained on StackOverflow:

https://stackoverflow.com/a/12702836/1178478

So the code that converts the Frame from the EasyAR streamer to a Bitmap is this:

Bitmap getBitmapFromFrameStreamer(cn.easyar.Frame frame) {
    Bitmap frame_bm = null;

    if (frame.images().size()>0) {
        int size   = frame.images().get(0).buffer().size();
        int width  = frame.images().get(0).width();
        int height = frame.images().get(0).height();
        int format = frame.images().get(0).format();
        if (size>0) {
            byte[] bytes = new byte[size];
            frame.images().get(0).buffer().copyTo(bytes, 0);

            // 2 == YUV_NV21
            if (format==2) {
                int[] pixels = convertYUV420_NV21toRGB8888(bytes, width, height);
                frame_bm = Bitmap.createBitmap(pixels, width, height, Bitmap.Config.ARGB_8888);
                if (frame_bm!=null)
                    Log.i(TAG, "frame bm: " + frame_bm.getWidth() + "x" + frame_bm.getHeight());
            }
        }
    }

    return frame_bm;
}

To use this method:

Frame frame = streamer.peek();
Bitmap frame_bm = getBitmapFromStreamer(frame);

The method convertYUV420_NV21toRGB8888() can be found on the StackOverflow linked before.

This code was added to the HelloAR.java from the HelloARVideo Android sample.

Welcome to EasyAR SDK Q&A, where you can ask questions and receive answers from other members of the community.
...