Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: V3 tensorflow plugin #1633

Draft
wants to merge 47 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
4b7fa85
tflite
mrousavy Jun 22, 2023
2648c3d
TF
mrousavy Jun 23, 2023
339f5e9
Tensorflow Plugin prototype
mrousavy Jun 23, 2023
de0dbb9
Update FrameProcessorRuntimeManager.mm
mrousavy Jun 23, 2023
966e2c7
Try calling model
mrousavy Jun 23, 2023
ff84d65
Data
mrousavy Jun 23, 2023
cb3de60
try accelerate
mrousavy Jun 23, 2023
383c7ed
Actually got it working lol wtf
mrousavy Jun 23, 2023
a1c24f4
Construct 2D array
mrousavy Jun 23, 2023
5f10c63
Add Metal and CoreML support
mrousavy Jun 26, 2023
1a2222f
Move up in scope
mrousavy Jun 26, 2023
6c3cde4
Float32
mrousavy Jun 26, 2023
fe39800
Move to `TensorflowPlugin.mm`
mrousavy Jun 26, 2023
36b6b98
free data
mrousavy Jun 26, 2023
1eff854
remove async
mrousavy Jun 26, 2023
5721263
Create buffer depending on output type
mrousavy Jun 28, 2023
fa47a85
Add support for Int8, Int16, Int32, and Float64
mrousavy Jun 28, 2023
72ef8c1
Update TensorflowPlugin.mm
mrousavy Jun 28, 2023
6d14868
Update TensorflowPlugin.mm
mrousavy Jun 28, 2023
e0b17f9
Log more
mrousavy Jun 28, 2023
bec364e
Update project.pbxproj
mrousavy Jun 29, 2023
7722b11
Convert
mrousavy Jun 29, 2023
bdf7d14
Correctly crop input frame
mrousavy Jun 29, 2023
31b9f84
Extract into HostObject
mrousavy Jun 29, 2023
4cd34a1
Loading type
mrousavy Jun 29, 2023
6635fec
Extract to `FrameResizer`
mrousavy Jun 29, 2023
963afc9
Clean
mrousavy Jun 29, 2023
697e6c8
Create `convertImageBufferToCGImage`
mrousavy Jun 29, 2023
35f4b3b
Add `inputs` and `outputs` property to Model
mrousavy Jun 30, 2023
eb2de8d
Expose delegate
mrousavy Jun 30, 2023
fe354ad
Async init Tensorflow
mrousavy Jun 30, 2023
53fac8c
Fix Result parsing
mrousavy Jun 30, 2023
de1d969
use all outputs
mrousavy Jun 30, 2023
deccdac
d
mrousavy Jun 30, 2023
01d2ecf
Pre-allocate ArrayBuffers
mrousavy Jun 30, 2023
61216f0
Try converting input to float
mrousavy Jun 30, 2023
32b8b9d
Try object detector
mrousavy Jun 30, 2023
00bb39b
Draw object bounding box
mrousavy Jun 30, 2023
978325e
Remove `_inputShape` class var
mrousavy Jun 30, 2023
ed9a336
Add `useTensorflowModel`
mrousavy Jul 3, 2023
f8e0b14
Update TensorflowPlugin.mm
mrousavy Jul 3, 2023
c6e25a5
Add `assetExts` again
mrousavy Jul 3, 2023
4dcc8e9
chore: Cleanup
mrousavy Jul 3, 2023
25b0d61
Update tsconfig.json
mrousavy Jul 3, 2023
aa6b8cf
chore: release 3.0.0-rc.3
mrousavy Jul 3, 2023
acb7e2a
Update podfile
mrousavy Jul 4, 2023
9166aa5
Drop builtInTrueDepthCamera
mrousavy Jul 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions cpp/JSITypedArray.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,12 @@ TypedArray<T>::TypedArray(jsi::Runtime &runtime, std::vector<ContentType<T>> dat
update(runtime, data);
}

template <TypedArrayKind T>
TypedArray<T>::TypedArray(jsi::Runtime &runtime, ContentType<T>* dataToCopy, size_t size)
: TypedArrayBase(runtime, size, T) {
updateUnsafe(runtime, dataToCopy, size);
}

template <TypedArrayKind T>
TypedArray<T>::TypedArray(TypedArrayBase &&base) : TypedArrayBase(std::move(base)) {}

Expand Down
1 change: 1 addition & 0 deletions cpp/JSITypedArray.h
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ class TypedArray : public TypedArrayBase {
public:
explicit TypedArray(TypedArrayBase &&base);
TypedArray(jsi::Runtime &runtime, size_t size);
TypedArray(jsi::Runtime &runtime, ContentType<T>* dataToCopy, size_t size);
TypedArray(jsi::Runtime &runtime, std::vector<ContentType<T>> data);
TypedArray(TypedArray &&) = default;
TypedArray &operator=(TypedArray &&) = default;
Expand Down
Binary file not shown.
1 change: 1 addition & 0 deletions example/ios/Podfile
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ target 'VisionCameraExample' do
)

pod 'VisionCamera', :path => '../..'
pod 'TensorFlowLiteObjC', :subspecs => ['Metal', 'CoreML']

post_install do |installer|
# https://github.com/facebook/react-native/blob/main/packages/react-native/scripts/react_native_pods.rb#L197-L202
Expand Down
Loading