current position:Home>Android uses cameraX to preview / take photos / record videos / analyze pictures / focus / switch cameras and other operations

Android uses cameraX to preview / take photos / record videos / analyze pictures / focus / switch cameras and other operations

2022-07-28 04:01:01Heiko-Android

1. CameraX framework

Look at the official documents CameraX framework
There is the following passage

Use CameraX, With the help of the name " Use cases " The abstract concept of interacting with the camera of the device .

  • preview : Accept... For displaying preview Surface, for example PreviewView
  • Picture analysis : To analyze ( For example, machine learning ) Provide CPU Accessible buffers
  • Picture shooting : Take and save pictures
  • Video shooting : adopt VideoCapture Shoot video and audio

Different use cases can be combined , It can also be active at the same time .
for example , Preview use cases can be added to the application , So that the user can view the picture entering the camera's field of view
Add picture analysis use cases , To determine whether the person in the photo is smiling
You can also add picture shooting use cases , In order to take photos when people smile

The first time I saw it , A face of meng ,“ Use cases ”, What the hell is it .

later , I studied it , know " Use cases " The original English text is called Use Case,CameraX Every operation in , Corresponding to a kind of UseCase

  • preview : Preview.java
  • Picture analysis : ImageAnalysis.java
  • Picture shooting : ImageCapture.java
  • Video shooting : VideoCapture.java

You can see , These classes are all inherited from UseCase.java Class

public final class Preview extends UseCase {
    
	//...
}
public final class ImageAnalysis extends UseCase {
    
	//...
}
public final class ImageCapture extends UseCase {
    
	//...
}
public final class VideoCapture extends UseCase {
    
	//...
}

Next, let's try to use .

2. Front operation

First , We need to build a new project , Then introduce dependency

// CameraX core library using the camera2 implementation
def camerax_version = "1.2.0-alpha02" //1.2.0-alpha02
// The following line is optional, as the core library is included indirectly by camera-camera2
implementation "androidx.camera:camera-core:${camerax_version}"
implementation "androidx.camera:camera-camera2:${camerax_version}"
// If you want to additionally use the CameraX Lifecycle library
implementation "androidx.camera:camera-lifecycle:${camerax_version}"
// If you want to additionally use the CameraX VideoCapture library
implementation "androidx.camera:camera-video:${camerax_version}"
// If you want to additionally use the CameraX View class
implementation "androidx.camera:camera-view:${camerax_version}"
// If you want to additionally add CameraX ML Kit Vision Integration
implementation "androidx.camera:camera-mlkit-vision:${camerax_version}"
// If you want to additionally use the CameraX Extensions library
implementation "androidx.camera:camera-extensions:${camerax_version}"

stay AndroidManifest.xml Add permissions to

<!-- Camera permissions -->
<uses-permission android:name="android.permission.CAMERA" />
<!-- With cameras -->
<uses-feature android:name="android.hardware.camera.any" />
<!-- Permission to store images or videos -->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<!-- Record audio permission -->
<uses-permission android:name="android.permission.RECORD_AUDIO" />

Don't forget to apply for permission

ActivityCompat.requestPermissions(
        this, arrayOf(
            Manifest.permission.CAMERA,
            Manifest.permission.WRITE_EXTERNAL_STORAGE,
            Manifest.permission.READ_EXTERNAL_STORAGE,
            Manifest.permission.RECORD_AUDIO
        ), 123
    )

3. preview : Preview.java

The first change activity_main.xml

<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:app="http://schemas.android.com/apk/res-auto" android:id="@+id/camera_container" android:layout_width="match_parent" android:layout_height="match_parent" android:background="@android:color/black">

    <androidx.camera.view.PreviewView android:id="@+id/previewView" android:layout_width="0dp" android:layout_height="0dp" app:layout_constraintBottom_toBottomOf="parent" app:layout_constraintLeft_toLeftOf="parent" app:layout_constraintRight_toRightOf="parent" app:layout_constraintTop_toTopOf="parent" />

</androidx.constraintlayout.widget.ConstraintLayout>

modify MainActivity.kt

class MainActivity : AppCompatActivity() {
    
    private lateinit var binding: ActivityMainBinding
    private lateinit var cameraProvider: ProcessCameraProvider
    private var preview: Preview? = null
    private var camera: Camera? = null

    override fun onCreate(savedInstanceState: Bundle?) {
    
        super.onCreate(savedInstanceState)
        binding = ActivityMainBinding.inflate(layoutInflater)
        setContentView(binding.root)

        //TODO  Permission application is omitted , See the article for details  " Front operation "  part 

        setUpCamera(binding.previewView)
    }

    private fun setUpCamera(previewView: PreviewView) {
    
        val cameraProviderFuture: ListenableFuture<ProcessCameraProvider> =
            ProcessCameraProvider.getInstance(this)
        cameraProviderFuture.addListener({
    
            try {
    
                cameraProvider = cameraProviderFuture.get()
                bindPreview(cameraProvider, previewView)
            } catch (e: Exception) {
    
                e.printStackTrace()
            }
        }, ContextCompat.getMainExecutor(this))
    }

    private fun bindPreview(
        cameraProvider: ProcessCameraProvider,
        previewView: PreviewView
    ) {
    
	    // Unbind all , prevent CameraProvider Repeatedly bind to Lifecycle Something goes wrong 
        cameraProvider.unbindAll()
        preview = Preview.Builder().build()
        camera = cameraProvider.bindToLifecycle(
            this,
            CameraSelector.DEFAULT_BACK_CAMERA, preview
        )
        preview?.setSurfaceProvider(previewView.surfaceProvider)
    }
}

Look at the effect
 Insert picture description here

4. Image analysis : ImageAnalysis.java

Image analysis use case ImageAnalysis Provide real-time image data for applications , We can perform image processing on these images 、 Computer vision or machine learning inference .

val imageAnalysis = ImageAnalysis.Builder()
    // enable the following line if RGBA output is needed.
    // .setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888)
    .setTargetResolution(Size(1280, 720))
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .build()
imageAnalysis.setAnalyzer(executor, ImageAnalysis.Analyzer {
     imageProxy ->
    val rotationDegrees = imageProxy.imageInfo.rotationDegrees
    // insert your code here.
    //  Here we deal with the analysis of pictures , For example, it can be parsed into QR code 
    ...
    // after done, release the ImageProxy object
    imageProxy.close()
})

Calling cameraProvider.bindToLifecycle() when , Pass in

cameraProvider.bindToLifecycle(
	this, CameraSelector.DEFAULT_BACK_CAMERA, preview, imageAnalysis
)

5. Taking pictures : ImageCapture.java

5.1 Photo only

here , We need to create a imageCapture

imageCapture = ImageCapture.Builder()
	.setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
	//.setTargetAspectRatio(screenAspectRatio)
	//.setTargetRotation(binding.previewView.display.rotation)
	.build()

then , Calling cameraProvider.bindToLifecycle() when , Pass in

camera = cameraProvider.bindToLifecycle(
    this,CameraSelector.DEFAULT_BACK_CAMERA, preview, imageCapture
)

increase takePicture() How to take photos

// Take pictures 
private fun takePicture() {
    
    imageCapture?.let {
     imageCapture ->
        val mainExecutor = ContextCompat.getMainExecutor(this)
        imageCapture.takePicture(mainExecutor, object : ImageCapture.OnImageCapturedCallback() {
    
            override fun onCaptureSuccess(image: ImageProxy) {
    
                super.onCaptureSuccess(image)
            }

            override fun onError(exception: ImageCaptureException) {
    
                super.onError(exception)
            }
        })

        //  Let the picture flash , Create the feeling of taking photos 
        // We can only change the foreground Drawable using API level 23+ API
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
    
            // Display flash animation to indicate that photo was captured
            binding.root.postDelayed({
    
                binding.root.foreground = ColorDrawable(Color.WHITE)
                binding.root.postDelayed(
                    {
     binding.root.foreground = null }, 50L
                )
            }, 100L)
        }
    }
}

5.2 Take pictures and save them to local storage

We can also take photos , Save to local storage

/** Helper function used to create a timestamped file */
private fun createFile(baseFolder: File, format: String, extension: String) =
    File(
        baseFolder, SimpleDateFormat(format, Locale.US)
            .format(System.currentTimeMillis()) + extension
    )

/** Use external media if it is available, our app's file directory otherwise */
fun getOutputDirectory(context: Context): File {
    
    val appContext = context.applicationContext
    val mediaDir = context.externalMediaDirs.firstOrNull()?.let {
    
        File(it, appContext.resources.getString(R.string.app_name)).apply {
     mkdirs() }
    }
    return if (mediaDir != null && mediaDir.exists())
        mediaDir else appContext.filesDir
}

companion object {
    
    private const val FILENAME = "yyyy-MM-dd-HH-mm-ss-SSS"
    private const val PHOTO_EXTENSION = ".jpg"
}

// Take photos and save them locally 
private fun takePictureSaveToDisk() {
    
    imageCapture?.let {
     imageCapture ->

        // Create output file to hold the image
        val photoFile = createFile(getOutputDirectory(this), FILENAME, PHOTO_EXTENSION)
        Log.i(TAG, "photoFile:$photoFile")

        // Setup image capture metadata
        val metadata = ImageCapture.Metadata().apply {
    

            // Mirror image when using the front camera
            isReversedHorizontal = lensFacing == CameraSelector.LENS_FACING_FRONT
        }

        // Create output options object which contains file + metadata
        val outputOptions = ImageCapture.OutputFileOptions.Builder(photoFile)
            .setMetadata(metadata)
            .build()

        // Setup image capture listener which is triggered after photo has been taken
        imageCapture.takePicture(
            outputOptions,
            ContextCompat.getMainExecutor(this),
            object : ImageCapture.OnImageSavedCallback {
    
                override fun onError(exc: ImageCaptureException) {
    
                    Log.e(TAG, "Photo capture failed: ${
      exc.message}", exc)
                }

                override fun onImageSaved(output: ImageCapture.OutputFileResults) {
    
                    val savedUri = output.savedUri ?: Uri.fromFile(photoFile)
                    Log.d(TAG, "Photo capture succeeded: $savedUri")

                    // Implicit broadcasts will be ignored for devices running API level >= 24
                    // so if you only target API level 24+ you can remove this statement
                    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) {
    
                        application.sendBroadcast(
                            Intent(android.hardware.Camera.ACTION_NEW_PICTURE, savedUri)
                        )
                    }

                    // If the folder selected is an external media directory, this is
                    // unnecessary but otherwise other apps will not be able to access our
                    // images unless we scan them using [MediaScannerConnection]
                    val mimeType = MimeTypeMap.getSingleton()
                        .getMimeTypeFromExtension(savedUri.toFile().extension)
                    MediaScannerConnection.scanFile(
                        application,
                        arrayOf(savedUri.toFile().absolutePath),
                        arrayOf(mimeType)
                    ) {
     _, uri ->
                        Log.d(TAG, "Image capture scanned into media store: $uri")
                    }
                }
            })

 		//  Let the picture flash , Create the feeling of taking photos 
        // We can only change the foreground Drawable using API level 23+ API
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
    
            // Display flash animation to indicate that photo was captured
            binding.root.postDelayed({
    
                binding.root.foreground = ColorDrawable(Color.WHITE)
                binding.root.postDelayed(
                    {
     binding.root.foreground = null }, 50L
                )
            }, 100L)
        }
    }
}

then , We can find this picture in the album , The real location of the picture is /storage/emulated/0/Android/media/ Your bag name / Project name / in .

6. video recording : VideoCapture.java

Video recording uses VideoCapture

videoCapture = VideoCapture.Builder()
	//.setTargetRotation(previewView.getDisplay().getRotation())
	.setVideoFrameRate(25)
	.setBitRate(3 * 1024 * 1024)
	.build()

Calling cameraProvider.bindToLifecycle() when , Pass in .

camera = cameraProvider.bindToLifecycle(
    this,CameraSelector.DEFAULT_BACK_CAMERA, preview, videoCapture 
)

It should be noted that ,videoCapture Can't and imageAnalysisimageCapture Use it together .
If these functions are integrated in the same page , You need to judge by the flag bit .

if (isVideo) {
    
    mCamera = cameraProvider.bindToLifecycle(this, cameraSelector,
            preview, videoCapture);
} else {
    
    mCamera = cameraProvider.bindToLifecycle(this, cameraSelector,
            preview, imageCapture, imageAnalysis);
}

Start recording

private val RECORDED_FILE_NAME = "recorded_video"
private val RECORDED_FILE_NAME_END = "video/mp4"

@SuppressLint("RestrictedApi")
private fun startRecording() {
    
	//TODO  Omit here RECORD_AUDIO、PERMISSION_GRANTED Judgment of authority 
	
    val contentValues = ContentValues()
    contentValues.put(
        MediaStore.MediaColumns.DISPLAY_NAME,
        RECORDED_FILE_NAME + "_" + System.currentTimeMillis()
    )
    contentValues.put(MediaStore.MediaColumns.MIME_TYPE, RECORDED_FILE_NAME_END)

    val outputFileOptions = VideoCapture.OutputFileOptions.Builder(
        getContentResolver(),
        MediaStore.Video.Media.EXTERNAL_CONTENT_URI, contentValues
    ).build()
    videoCapture.startRecording(
        outputFileOptions,
        ContextCompat.getMainExecutor(this),
        object : VideoCapture.OnVideoSavedCallback {
    
            override fun onVideoSaved(outputFileResults: VideoCapture.OutputFileResults) {
    
                Log.i(TAG, " Video saved successfully :${
      outputFileResults.savedUri}")
            }

            override fun onError(
                videoCaptureError: Int,
                message: String,
                cause: Throwable?
            ) {
    
                Log.i(TAG, " When something goes wrong  cause:$cause")
            }
        }
    )
}

Stop video recording

videoCapture.stopRecording()

When we execute Stop video recording after , You can see an additional recorded video in the album .

It introduces CameraX Some commonly used UseCase, Let's take a look at CameraX Other functions in .

7. Switch front and rear cameras

We used cameraProvider.bindToLifecycle() When , One parameter is CameraSelector.
CameraX By default, we are provided with front camera and rear camera CameraSelector

public final class CameraSelector {
    
    @NonNull
    public static final CameraSelector DEFAULT_FRONT_CAMERA =
            new CameraSelector.Builder().requireLensFacing(LENS_FACING_FRONT).build();

    @NonNull
    public static final CameraSelector DEFAULT_BACK_CAMERA =
            new CameraSelector.Builder().requireLensFacing(LENS_FACING_BACK).build();

	//...
}

When we go to switch cameras , Just call again bindPreview Method , Introduce new cameraSelector It's worth it

private fun bindPreview(
        cameraProvider: ProcessCameraProvider,
        previewView: PreviewView,
        cameraSelector : CameraSelector
    ) {
    
        //  Unbind all , prevent CameraProvider Repeatedly bind to Lifecycle Something goes wrong 
        cameraProvider.unbindAll()
        preview = Preview.Builder().build()
        camera = cameraProvider.bindToLifecycle(
            this,
            cameraSelector, preview
        )
        preview?.setSurfaceProvider(previewView.surfaceProvider)
    }

CameraX It also provides us with judgment In front of / Rear camera Method exists or not

/** Returns true if the device has an available back camera. False otherwise */
private fun hasBackCamera(): Boolean {
    
    return cameraProvider?.hasCamera(CameraSelector.DEFAULT_BACK_CAMERA) ?: false
}

/** Returns true if the device has an available front camera. False otherwise */
private fun hasFrontCamera(): Boolean {
    
    return cameraProvider?.hasCamera(CameraSelector.DEFAULT_FRONT_CAMERA) ?: false
}

Look at the effect
 Insert picture description here

8. focusing

When you click androidx.camera.view.PreviewView When , To call CameraX Focusing method startFocusAndMetering() Just fine .

stay onCreate() Add the following code to

binding.previewView.setOnTouchListener {
     view, event ->
    val action = FocusMeteringAction.Builder(
        binding.previewView.getMeteringPointFactory()
            .createPoint(event.getX(), event.getY())
    ).build();
    showTapView(event.x.toInt(), event.y.toInt())
    camera?.getCameraControl()?.startFocusAndMetering(action)
    true
}

increase showTapView()

private fun showTapView(x: Int, y: Int) {
    
    val popupWindow = PopupWindow(
        ViewGroup.LayoutParams.WRAP_CONTENT,
        ViewGroup.LayoutParams.WRAP_CONTENT
    )
    val imageView = ImageView(this)
    imageView.setImageResource(R.drawable.ic_focus_view)
    popupWindow.contentView = imageView
    popupWindow.showAsDropDown(binding.previewView, x, y)
    binding.previewView.postDelayed({
     popupWindow.dismiss() }, 600)
    binding.previewView.playSoundEffect(SoundEffectConstants.CLICK)
}

Look at the effect
 Insert picture description here

9. this paper Demo download

Source code of this article Demo See : Android CameraX Demo : Implementation preview / Taking pictures / record video / Picture analysis / focusing / Switch camera and other operations

Reference resources
Android Developer | CameraX
Here comes the introduction of new technology ,CameraX Unify river's lake ?

copyright notice
author[Heiko-Android],Please bring the original link to reprint, thank you.
https://en.hqmana.com/2022/197/202207132102215654.html

Random recommended