current position:Home>Machine learning makes photo translation easier: java+ Huawei ml Kit

Machine learning makes photo translation easier: java+ Huawei ml Kit

2022-07-29 05:50:41Quantify NPC

brief introduction

Photo translation combines both text recognition and Translation . As long as the user turns on the camera and takes pictures of the foreign characters that need to be recognized , that will do
Translate into your mother tongue .

In reality , If you go to a place where you don't know the language , The probability may encounter the following problems :

Dream menu

 Insert picture description here

Magic road signs

 Insert picture description here

There are a wide range of goods in the mall

 Insert picture description here

Photo translator helps you

Let's first look at the effect display
meanwhile po Source code ( find Photo-translate):
https://github.com/HMS-MLKit/HUAWEI-HMS-MLKit-Sample
 Insert picture description here

Introduction to basic functions ( free

Today I'd like to introduce you , How to develop a photo translation service . In short, it only takes two big steps to complete the development of the photo translation app :

Text recognition

 Insert picture description here
Take a picture first to get the picture , Then send the acquired image frames to Huawei HMS ML Kit Text recognition services do text recognition
Huawei text recognition service also provides offline SDK( End side ) And cloud side , Free end-to-side real-time detection , The type and accuracy of cloud side recognition are higher . In this actual battle, we use the capabilities provided by the cloud side .

Text recognition features Text recognition features
End side Supporting 、 Japan 、 Han 、 Latin characters
Cloud side in 、 Britain 、 Law 、 In the west 、 Thai, etc 19 Languages
Tilt recognition 30 It can still be recognized in the case of degree inclination
Curved text support Support 45 Degree bending can still be successfully identified
Text tracking The end side supports tracking

See the official website of Huawei developer Alliance for details :
https://developer.huawei.com/consumer/cn/doc/development/HMSCore-Guides/text-recognition-0000001050040053

translate

 Insert picture description here
Send the recognized text to Huawei HMS MLKit Translation services do text translation , You can get the results you want after translation .
Translation is a service provided by the cloud side .

Text translation features specifications (HMS 4.0)
Multilingual 7 Languages in 、 Britain 、 Law 、 In the west 、 soil 、 o 、 Thai
Time delay 300ms/100 word
BLEU value >30
Dynamic term configuration Support

See the official website of Huawei developer Alliance for details :
https://developer.huawei.com/consumer/cn/doc/development/HMSCore-Guides/text-translation-0000001050040079

Photo translation APP Developing actual combat

1 The development of preparation

Due to the use of cloud side Services , You need to register a developer account with Huawei's developer alliance , And open these services in the cloud , I won't go into details here , Directly according to the official AppGallery Connect To configure 、 The operation steps of opening the service can be carried out :
Registered developer , For service reference, please stamp :

1.1 At the project level gradle Add Huawei maven warehouse

open AndroidStudio Project level build.gradle file .

Incrementally add the following maven Address :

buildscript {
    repositories {        
        maven {url 'http://developer.huawei.com/repo/'}
    }    
}
allprojects {
    repositories {       
        maven { url 'http://developer.huawei.com/repo/'}
    }
}
1.2 At the application level build.gradle Inside plus SDK rely on

Integrate SDK.( Due to the use of cloud side capabilities , Just introduce SDK Basic package is enough )

dependencies{  
    implementation 'com.huawei.hms:ml-computer-vision:1.0.2.300'
    implementation 'com.huawei.hms:ml-computer-translate:1.0.2.300'
}

1.3 stay AndroidManifest.xml Apply for camera and storage permissions in the file

<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.camera.autofocus" />

2 Key steps in code development
2.1 Dynamic permission application
private static final int CAMERA_PERMISSION_CODE = 1; 
@Override 
public void onCreate(Bundle savedInstanceState) { 
    // Checking camera permission   
   if (!allPermissionsGranted()) {
       getRuntimePermissions();
   }
} 
2.2 Create a cloud side text analyzer . You can use the text detection configurator “MLRemoteTextSetting” Create a text parser .
MLRemoteTextSetting setting = (new MLRemoteTextSetting.Factory()).
         setTextDensityScene(MLRemoteTextSetting.OCR_LOOSE_SCENE).create();
this.textAnalyzer = MLAnalyzerFactory.getInstance().getRemoteTextAnalyzer(setting);
2.3 adopt android.graphics.Bitmap establish “MLFrame” Object is used by the analyzer to detect pictures .
MLFrame mlFrame = new MLFrame.Creator().setBitmap(this.originBitmap).create();
2.4 call “asyncAnalyseFrame ” Method for text detection .
Task<MLText> task = this.textAnalyzer.asyncAnalyseFrame(mlFrame);
            task.addOnSuccessListener(new OnSuccessListener<MLText>() {
                @Override
                public void onSuccess(MLText mlText) {
                    // Transacting logic for segment success.
                    if (mlText != null) {
                        RemoteTranslateActivity.this.remoteDetectSuccess(mlText);
                    } else {
                        RemoteTranslateActivity.this.displayFailure();
                    }
                }
            }).addOnFailureListener(new OnFailureListener() {
                @Override
                public void onFailure(Exception e) {
                    // Transacting logic for segment failure.
                    RemoteTranslateActivity.this.displayFailure();
                    return;
                }
            });
2.5 Create a text translator . You can customize the parameter class through the text translator “MLRemoteTranslateSetting” Create translator .
MLRemoteTranslateSetting.Factory factory = new MLRemoteTranslateSetting
                .Factory()
                // Set the target language code. The ISO 639-1 standard is used.
                .setTargetLangCode(this.dstLanguage);
        if (!this.srcLanguage.equals("AUTO")) {
            // Set the source language code. The ISO 639-1 standard is used.
            factory.setSourceLangCode(this.srcLanguage);
        }
        this.translator = MLTranslatorFactory.getInstance().getRemoteTranslator(factory.create());
2.6 call “asyncAnalyseFrame ” Methods to translate the content obtained by text recognition .
final Task<String> task = translator.asyncTranslate(this.sourceText);
        task.addOnSuccessListener(new OnSuccessListener<String>() {
            @Override
            public void onSuccess(String text) {
                if (text != null) {
                    RemoteTranslateActivity.this.remoteDisplaySuccess(text);
                } else {
                    RemoteTranslateActivity.this.displayFailure();
                }
            }
        }).addOnFailureListener(new OnFailureListener() {
            @Override
            public void onFailure(Exception e) {
                RemoteTranslateActivity.this.displayFailure();
            }
        });
2.7 Release resources after translation .
if (this.textAnalyzer != null) {
            try {
                this.textAnalyzer.close();
            } catch (IOException e) {
                SmartLog.e(RemoteTranslateActivity.TAG, "Stop analyzer failed: " + e.getMessage());
            }
        }
        if (this.translator != null) {
            this.translator.stop();
        }

3 Source code

Old rules , Simple little Demo The source code has been uploaded Github,github Please stamp the source address ( The project catalogue is :Photo-Translate), You can do scene based optimization for reference .
https://github.com/HMS-MLKit/HUAWEI-HMS-MLKit-Sample

4 The conclusion is

The small program demonstrated to you this time APP Huawei is also used in the development HMS ML Kit Two cloud side capabilities , Ability of character recognition and Translation , Huawei's character recognition and translation can also help developers do many other interesting and powerful functions , such as :

【 General text recognition 】

1、 Bus license plate character recognition
2、 Text recognition in document reading scene

【 Card type text recognition 】

1、 The card number of the bank card can be recognized through character recognition , Used in bank card binding and other scenarios
2、 Of course, in addition to identifying bank cards , It can also identify various card numbers in life , Such as membership card 、 coupon card
3、 In addition, it can also realize ID card 、 Identification of certificate numbers such as Hong Kong and Macao passes

【 translate 】

1、 Translation of road signs
2、 To translate documents
3、 Web translation , For example, identify the language type of the website comment area and translate it into the language of the corresponding country ;
4、 Translation of Haitao product introduction
5、 Restaurant order menu translation

5 Huawei MLkit Other functions

 Insert picture description here
Welcome to the official website
https://developer.huawei.com/consumer/cn/doc/development/HMSCore-Guides/service-introduction-0000001050040017
Sample code reference :( Don't use wechat scanning )
https://developer.huawei.com/consumer/cn/doc/development/HMSCore-Examples-V5/sample-code-0000001050265470-V5
Huawei's application market will have the opportunity to get free traffic promotion

copyright notice
author[Quantify NPC],Please bring the original link to reprint, thank you.
https://en.hqmana.com/2022/210/202207290519021023.html