add tflite generative ai example #455

zehuiw · 2023-04-25T18:23:59Z

Add TFLite generative AI Android example
https://codelabs.developers.google.com/kerasnlp-tflite#0

JunyoungLim

Thanks for the great demo!

In this review, I pretended I'm the first time reader going through the code and pointed out any parts that are unclear (so a lot of nit-comments, but most of them are code suggestion that you can just accept and then commit), and a few enhancement that we can bring.

JunyoungLim · 2023-04-27T22:50:04Z

lite/examples/generative_ai/android/app/libs/build_aar/build_aar.sh

@@ -0,0 +1,24 @@
+#! /bin/bash
+


nit: how about exiting on failure and reporting the error message to the user before proceeding?

Suggested change

set -e

JunyoungLim · 2023-04-27T22:52:42Z

lite/examples/generative_ai/android/app/libs/build_aar/build_aar.sh

+if [ $? -eq 0 ]; then
+    # Print a message
+    echo "Please find the aar file: tensorflow_text/bazel-bin/tensorflow_text/tftext_tflite_flex.aar"
+fi


nit: add an error message for the user? Up to the code owner :)

Suggested change

fi

else

echo "build_aar.sh has failed. Please find the error message above and address it before proceeding."

fi

JunyoungLim · 2023-04-27T23:20:40Z

lite/examples/generative_ai/android/settings.gradle.kts

+    }
+}
+
+rootProject.name = "Google Tensorflow Demo"


Suggested change

rootProject.name = "Google Tensorflow Demo"

rootProject.name = "Google TensorFlow Demo"

JunyoungLim · 2023-05-01T18:47:13Z

lite/examples/generative_ai/android/README.md

+
+For this demonstration, we will use KerasNLP to get the GPT-2 model. KerasNLP is a library that contains state-of-the-art pretrained models for natural language processing tasks, and can support users through their entire development cycle. You can see the list of models available in the [KerasNLP repository](https://github.com/keras-team/keras-nlp/tree/master/keras_nlp/models). The workflows are built from modular components that have state-of-the-art preset weights and architectures when used out-of-the-box and are easily customizable when more control is needed. Creating the GPT-2 model can be done with the following steps:
+
+```


I tried reading the markdown in the user preview mode, and I think we might need syntax highlighting as below.

Please do so for all the code blocks.

Suggested change

```

```python

JunyoungLim · 2023-05-01T18:52:26Z

lite/examples/generative_ai/android/README.md

+)
+```
+
+You can check out the full GPT-2 model implementation on GitHub.


Is there a link we want to attach here for the full GPT-2 model implementation on GitHub part?

JunyoungLim · 2023-05-01T19:40:41Z

lite/examples/generative_ai/android/how-to-build.md

+*	Open Android Studio, and from the Welcome screen, select Open an existing Android Studio project.
+*	From the Open File or Project window that appears, navigate to and select the `lite/examples/generative_ai/android` directory from wherever you cloned the TensorFlow Lite sample GitHub repo.
+*	You may also need to install various platforms and tools according to error messages.
+*	Rename the converted `.tflite model` to `autocomplete.tflite` and copy it into `app/src/main/assets/` folder
+*	Select menu `Build -> Make Project` to build the app. (Ctrl+F9, depending on your version).
+*	Click menu `Run -> Run 'app'`. (Shift+F10, depending on your version)


Suggested change

* Open Android Studio, and from the Welcome screen, select Open an existing Android Studio project.

* From the Open File or Project window that appears, navigate to and select the `lite/examples/generative_ai/android` directory from wherever you cloned the TensorFlow Lite sample GitHub repo.

* You may also need to install various platforms and tools according to error messages.

* Rename the converted `.tflite model` to `autocomplete.tflite` and copy it into `app/src/main/assets/` folder

* Select menu `Build -> Make Project` to build the app. (Ctrl+F9, depending on your version).

* Click menu `Run -> Run 'app'`. (Shift+F10, depending on your version)

* Open Android Studio, and from the Welcome screen, select Open an existing Android Studio project.

* From the `Open File or Project` window that appears, navigate to and select the `lite/examples/generative_ai/android` directory from wherever you cloned the TensorFlow Lite sample GitHub repo.

* You may also need to install various platforms and tools according to error messages.

* Rename the converted `.tflite` model to `autocomplete.tflite` and copy it into `app/src/main/assets/` folder.

* Select menu `Build -> Make Project` to build the app (Ctrl+F9, depending on your version).

* Click menu `Run -> Run 'app'` (Shift+F10, depending on your version)`

lite/examples/generative_ai/android/how-to-build.md

JunyoungLim · 2023-05-01T20:00:30Z

...android/app/src/main/java/com/google/tensorflowdemo/data/autocomplete/AutoCompleteService.kt

I don't think adding method comments for all is not necessary, but for this file, adding method comments for all the methods in the file will be helpful for the first-time readers (as this file seems to be the most important main one).

Two requests:

adding a doc-string method comment

adding inline comments within the implementation for easier understanding

Thanks!

JunyoungLim · 2023-05-01T20:04:39Z

...android/app/src/main/java/com/google/tensorflowdemo/data/autocomplete/AutoCompleteService.kt

+
+        interpreter.run(input, outputBuffer)
+
+        outputBuffer.flip().position(12)


Can we make this 12 as a global variable and add comment to describe where the number is coming from?

JunyoungLim · 2023-05-01T20:09:36Z

...android/app/src/main/java/com/google/tensorflowdemo/data/autocomplete/AutoCompleteService.kt

+        minWordCount = 5,
+        maxWordCount = min(50, MAX_INPUT_WORD_COUNT),
+        initialWordCount = 20


Could you add some explanation on where the numbers come from? Thanks!

zehuiw · 2023-05-02T23:43:09Z

Thank you so much for your reviews @JunyoungLim ! The team revised the code in the latest commit. Tested in Android Simulator.

JunyoungLim

Thanks for addressing them all!

Just three more comments!

JunyoungLim · 2023-05-03T06:53:13Z

lite/examples/generative_ai/android/README.md

+gpt2_lm = 
+keras_nlp.models.GPT2CausalLM.from_preset(
+"gpt2_base_en", 
+preprocessor=gpt2_preprocessor
+)


Suggested change

gpt2_lm =

keras_nlp.models.GPT2CausalLM.from_preset(

"gpt2_base_en",

preprocessor=gpt2_preprocessor

)

gpt2_lm = keras_nlp.models.GPT2CausalLM.from_preset(

"gpt2_base_en",

preprocessor=gpt2_preprocessor,

)

JunyoungLim · 2023-05-03T06:58:21Z

...id/app/src/main/java/com/google/tensorflowdemo/ui/screens/autocomplete/AutoCompleteScreen.kt

+            inputValue = inputValue,
+            onInputValueChange = {},
+            inputEnabled = true,
+            inputConfiguration = AutoCompleteInputConfiguration(5, 50, 20),


I have seen another AutoCompleteInputConfiguration(5, 50, 20) in other files, do they all need to be the same across the app? If so, would it be better if we factor them out and keep it in a single place and import them? (I will leave this to the code owner; I'm suggesting this in case it will make the code maintenance/upgrade easier later).

JunyoungLim · 2023-05-03T06:59:52Z

...ain/java/com/google/tensorflowdemo/ui/screens/autocomplete/components/WindowSizeSelection.kt

+fun PreviewWindowSizeSelection() {
+    TensorFlowDemoTheme {
+        WindowSizeSelection(
+            inputConfiguration = AutoCompleteInputConfiguration(5, 50, 20),


For example, here I see another AutoCompleteInputConfiguration(5, 50, 20)

zehuiw · 2023-05-03T20:57:04Z

Thank you so much @JunyoungLim ! Resolved in the latest commit.

JunyoungLim

LGTM!

add tflite generative ai example

87d0959

zehuiw requested review from khanhlvg and lu-wang-g as code owners April 25, 2023 18:24

google-ml-butler bot added the size:XL CL Change Size: Extra Large label Apr 25, 2023

google-ml-butler bot assigned gbaned Apr 25, 2023

google-ml-butler bot added the awaiting review label Apr 25, 2023

add ipynb link for generating tflite models

05fa130

JunyoungLim suggested changes May 1, 2023

View reviewed changes

improve documentations and code quanlity

011bc73

zehuiw closed this May 2, 2023

google-ml-butler bot removed the awaiting review label May 2, 2023

zehuiw reopened this May 2, 2023

JunyoungLim reviewed May 3, 2023

View reviewed changes

add default input configurations

0dd83ec

JunyoungLim approved these changes May 3, 2023

View reviewed changes

google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull labels May 3, 2023

kokoro-team removed the kokoro:force-run Tests on submitted change label May 3, 2023

miaout17 approved these changes May 5, 2023

View reviewed changes

google-ml-butler bot added the kokoro:force-run Tests on submitted change label May 5, 2023

kokoro-team removed the kokoro:force-run Tests on submitted change label May 5, 2023

terryheo merged commit 1ca6132 into tensorflow:master May 8, 2023

google-ml-butler bot removed the ready to pull label May 8, 2023

miaout17 mentioned this pull request May 10, 2023

add tflite example #457

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add tflite generative ai example #455

add tflite generative ai example #455

zehuiw commented Apr 25, 2023

JunyoungLim left a comment

JunyoungLim Apr 27, 2023

JunyoungLim Apr 27, 2023

JunyoungLim Apr 27, 2023

JunyoungLim May 1, 2023

JunyoungLim May 1, 2023

JunyoungLim May 1, 2023

JunyoungLim May 1, 2023

JunyoungLim May 1, 2023

JunyoungLim May 1, 2023

zehuiw commented May 2, 2023

JunyoungLim left a comment

JunyoungLim May 3, 2023

JunyoungLim May 3, 2023

JunyoungLim May 3, 2023

zehuiw commented May 3, 2023

JunyoungLim left a comment

-fi
+else
+    echo "build_aar.sh has failed. Please find the error message above and address it before proceeding."
+fi

	rootProject.name = "Google Tensorflow Demo"
	rootProject.name = "Google TensorFlow Demo"


		For this demonstration, we will use KerasNLP to get the GPT-2 model. KerasNLP is a library that contains state-of-the-art pretrained models for natural language processing tasks, and can support users through their entire development cycle. You can see the list of models available in the [KerasNLP repository](https://github.com/keras-team/keras-nlp/tree/master/keras_nlp/models). The workflows are built from modular components that have state-of-the-art preset weights and architectures when used out-of-the-box and are easily customizable when more control is needed. Creating the GPT-2 model can be done with the following steps:

		```


		interpreter.run(input, outputBuffer)

		outputBuffer.flip().position(12)

add tflite generative ai example #455

add tflite generative ai example #455

Conversation

zehuiw commented Apr 25, 2023

JunyoungLim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zehuiw commented May 2, 2023

JunyoungLim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zehuiw commented May 3, 2023

JunyoungLim left a comment

Choose a reason for hiding this comment