QR Scanning using CameraX
Recently I had to implement QR code scanning in Simple using CameraX and zxing (we are using zxing to decode the image and read the QR code). Thanks to CameraX it is much easier to implement camera APIs and use it to read the QR code, you might be wondering “what are the benefits of using CameraX?” For starters consistent and easy to use camera APIs, well-tested library with various OEMs and devices.
Let’s Code
You can find the example project here
Before we jump into code let’s look at what we need to do.
-
Show camera preview
-
Analyse image to read the QR code with zxing
In this article, we will take a look at setting up ImageAnalysis and using zxing to read the QR code. You can refer to CameraX official guide to see how to set up the camera preview using PreviewView.
Let’s start by adding zxing dependency. (We are using version 3.3.0 of the library because version 3.4.0 requires Java 8 or above)
implementation 'com.google.zxing:android-core:3.3.0'
Creating an Image Analyzer
The image analysis use case in CameraX provides the app with CPU accessible image to perform image processing, computer vision or machine learning.
We start by extending ImageAnalysis.Analyzer to create QrCodeAnalyzer.
import androidx.camera.core.ImageAnalysis
import androidx.camera.core.ImageProxy
class QrCodeAnalyzer : ImageAnalysis.Analyzer {
override fun analyze(image: ImageProxy) {
}
}
ImageAnalysis.Analyzer#analyze is called for every frame and this is where we will analyse the image and let zxing detect and read the QR code. During analyze we get ImageProxy as a parameter, this will give us access to the image information like format, width, height, planes, rotation degrees. Internally ImageProxy uses ImageReader to get the Image, ImageReader uses YUV format unless changed. So first let's verify that we are getting a YUV format image because based on that our zxing implementation may vary.
import android.graphics.ImageFormat.*
import android.os.Build
import android.util.Log
import androidx.camera.core.ImageAnalysis
import androidx.camera.core.ImageProxy
class QrCodeAnalyzer : ImageAnalysis.Analyzer {
private val yuvFormats = mutableListOf(YUV_420_888)
init {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
yuvFormats.addAll(listOf(YUV_422_888, YUV_444_888))
}
}
override fun analyze(image: ImageProxy) {
// We are using YUV format because, ImageProxy internally uses ImageReader to get the image
// by default ImageReader uses YUV format unless changed.
// https://developer.android.com/reference/androidx/camera/core/ImageProxy.html#getImage()
// https://developer.android.com/reference/android/media/Image.html#getFormat()
if (image.format !in yuvFormats) {
Log.e("QRCodeAnalyzer", "Expected YUV, now = ${image.format}")
return
}
}
}
To decode the image and read the QR we need a Reader, we will be using MultiFormatReader. By default it supports reading all barcode formats, since we only want a QR code scanner we can use hints to tell the Reader which formats to decode.
private val reader = MultiFormatReader().apply {
val map = mapOf(
DecodeHintType.POSSIBLE_FORMATS to arrayListOf(BarcodeFormat.QR_CODE)
)
setHints(map)
}
Now that we have our reader, let's set up the rest of the code that is required to decode the image. First, we need to get the Y plane from our image. (Since we our image format is YUV, it will be the first plane)
private fun ByteBuffer.toByteArray(): ByteArray {
rewind()
val data = ByteArray(remaining())
get(data)
return data
}
override fun analyze(image: ImageProxy) {
val data = image.planes[0].buffer.toByteArray()
}
Then we use the data to create a PlanarYUVLuminanceSource it essentially gives us greyscale luminance values for any pixel format where Y channel is planar and appears first.
override fun analyze(image: ImageProxy) {
//——
val data = image.planes[0].buffer.toByteArray()
val source = PlanarYUVLuminanceSource(
data,
image.width,
image.height,
0,
0,
image.width,
image.height,
false
)
}
Now all we have to do is take this PlanarYUVLuminanceSource and construct a BinaryBitmap so that our Reader can decode the QR code. To construct a BinaryBitmap we need to pass it a Binarizer which takes in the luminance data and convert to 1 bit data. We are using HybridBinarizer (mainly because it's recommended by zxing).
override fun analyze(image: ImageProxy) {
val data = image.planes[0].buffer.toByteArray()
val source = PlanarYUVLuminanceSource(
data,
image.width,
image.height,
0,
0,
image.width,
image.height,
false
)
val binaryBitmap = BinaryBitmap(HybridBinarizer(source))
try {
// Whenever reader fails to detect a QR code in image
// it throws NotFoundException
val result = reader.decode(binaryBitmap)
Log.d("QRCodeAnalyzer", result.text)
} catch (e: NotFoundException) {
e.printStackTrace()
}
image.close()
}
Finally we close the image so that further images can be processed. If we don't close the image, further image may not be sent to analyse, this is an issue when the first frame camera recognises may not have a QR code. So we have to keep sending image frames until the QR code is recognised.
Here is the final code for the QrCodeAnalyzer
import android.graphics.ImageFormat.*
import android.os.Build
import android.util.Log
import androidx.camera.core.ImageAnalysis
import androidx.camera.core.ImageProxy
import com.google.zxing.*
import com.google.zxing.common.HybridBinarizer
import java.nio.ByteBuffer
private fun ByteBuffer.toByteArray(): ByteArray {
rewind()
val data = ByteArray(remaining())
get(data)
return data
}
class QrCodeAnalyzer(
private val onQrCodesDetected: (qrCode: Result) -> Unit
) : ImageAnalysis.Analyzer {
private val yuvFormats = mutableListOf(YUV_420_888)
init {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
yuvFormats.addAll(listOf(YUV_422_888, YUV_444_888))
}
}
private val reader = MultiFormatReader().apply {
val map = mapOf(
DecodeHintType.POSSIBLE_FORMATS to arrayListOf(BarcodeFormat.QR_CODE)
)
setHints(map)
}
override fun analyze(image: ImageProxy) {
// We are using YUV format because, ImageProxy internally uses ImageReader to get the image
// by default ImageReader uses YUV format unless changed.
if (image.format !in yuvFormats) {
Log.e("QRCodeAnalyzer", "Expected YUV, now = ${image.format}")
return
}
val data = image.planes[0].buffer.toByteArray()
val source = PlanarYUVLuminanceSource(
data,
image.width,
image.height,
0,
0,
image.width,
image.height,
false
)
val binaryBitmap = BinaryBitmap(HybridBinarizer(source))
try {
// Whenever reader fails to detect a QR code in image
// it throws NotFoundException
val result = reader.decode(binaryBitmap)
onQrCodesDetected(result)
} catch (e: NotFoundException) {
e.printStackTrace()
}
image.close()
}
}
Now that we have our ImageAnalysis.Analyzer, we just need to set the use case to the ProcessCameraProvider . For that we create a ImageAnalysis use case and set our QrCodeAnalyzer. We are also setting a backpressure strategy incase image is not closed and a new frame is available.
val imageAnalysis = ImageAnalysis.Builder()
.setTargetResolution(Size(previewView.width, previewView.height))
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.build()
.also {
it.setAnalyzer(cameraExecutor, QrCodeAnalyzer { qrResult ->
previewView.post {
Log.d("QRCodeAnalyzer", "Barcode scanned: ${qrResult.text}")
finish()
}
})
}
cameraProvider.bindToLifecycle(this, cameraSelector, preview, imageAnalysis)
That's it, with just a few lines of code we have a working QR code scanner with CameraX and zxing.
Things to keep in mind
-
If you have issues with the image rotation,
ImageProxyprovides us with rotation degrees which can be used to adjust the frame/image before decoding. -
If you use a different image format than YUV, you may need to switch to a different a
LuminanceSource. -
If you found a mistake or something to improve upon, please let me know.