Skip to content

Commit

Permalink
[MI-3490] Added the logic of streaming files from MS Teams and upload…
Browse files Browse the repository at this point in the history
…ing directly to MM without occupying memory (#298)

* [MI-3490] Added the logic of streaming files from MS Teams and uploading directly to MM without occupying memory
Updated the Mattermost server version being used in go.mod to 7.8.10
Modified the existing client functions and created new functions to handle the usecase
Added the logic to download whole file into memory if file size is less than or equal to 20 MB
Added the logic to stream file from MS Teams and upload to Mattermost in chunks of size 10 MB

* [MI-3490] Created two new config settings for maxSizeForCompleteDownload and bufferSizeForStreaming
Used the two new config settings for streaming logic
Modified the interfaces of PluginIFace and Client
Modified some interfaces in test cases as well

* [MI-3490] Updated the minimum supported MM version and fixed the failing unit tests

* [MI-3490] Review fixes: Modified some comments and refactored some code
  • Loading branch information
manojmalik20 authored Sep 13, 2023
1 parent 224f476 commit 0eb2c41
Show file tree
Hide file tree
Showing 14 changed files with 360 additions and 189 deletions.
38 changes: 19 additions & 19 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ require (
github.com/gorilla/mux v1.8.0
github.com/gosimple/slug v1.13.1
github.com/jmoiron/sqlx v1.3.5
github.com/mattermost/mattermost-plugin-api v0.1.1
github.com/mattermost/mattermost-server/v6 v6.0.0-20221012175353-8cb6718a9bcc
github.com/mattermost/mattermost-plugin-api v0.1.4
github.com/mattermost/mattermost-server/v6 v6.0.0-20230901142735-ccceee77061a
github.com/mattn/godown v0.0.1
github.com/microsoft/kiota-abstractions-go v1.2.0
github.com/microsoftgraph/msgraph-sdk-go v1.16.0
Expand Down Expand Up @@ -49,40 +49,40 @@ require (
github.com/go-asn1-ber/asn1-ber v1.5.4 // indirect
github.com/go-logr/logr v1.2.4 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-sql-driver/mysql v1.6.0 // indirect
github.com/go-sql-driver/mysql v1.7.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v4 v4.5.0 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/uuid v1.3.1 // indirect
github.com/gopherjs/gopherjs v1.17.2 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/gosimple/unidecode v1.0.1 // indirect
github.com/graph-gophers/graphql-go v1.4.0 // indirect
github.com/hashicorp/go-hclog v1.2.2 // indirect
github.com/hashicorp/go-plugin v1.4.4 // indirect
github.com/graph-gophers/graphql-go v1.5.1-0.20230110080634-edea822f558a // indirect
github.com/hashicorp/go-hclog v1.4.0 // indirect
github.com/hashicorp/go-plugin v1.4.8 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.15.9 // indirect
github.com/klauspost/cpuid/v2 v2.1.0 // indirect
github.com/klauspost/compress v1.15.14 // indirect
github.com/klauspost/cpuid/v2 v2.2.3 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/lann/builder v0.0.0-20180802200727-47ae307949d0 // indirect
github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0 // indirect
github.com/lib/pq v1.10.6 // indirect
github.com/lib/pq v1.10.7 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mattermost/go-i18n v1.11.1-0.20211013152124-5c415071e404 // indirect
github.com/mattermost/ldap v0.0.0-20201202150706-ee0e6284187d // indirect
github.com/mattermost/logr/v2 v2.0.15 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/mattn/go-isatty v0.0.17 // indirect
github.com/mattn/go-runewidth v0.0.14 // indirect
github.com/microsoft/kiota-authentication-azure-go v1.0.0 // indirect
github.com/microsoft/kiota-http-go v1.1.0 // indirect
github.com/microsoft/kiota-serialization-form-go v1.0.0 // indirect
github.com/microsoft/kiota-serialization-json-go v1.0.4 // indirect
github.com/microsoft/kiota-serialization-multipart-go v1.0.0 // indirect
github.com/microsoft/kiota-serialization-text-go v1.0.0 // indirect
github.com/minio/md5-simd v1.1.2 // indirect
github.com/minio/minio-go/v7 v7.0.34 // indirect
github.com/minio/minio-go/v7 v7.0.45 // indirect
github.com/minio/sha256-simd v1.0.0 // indirect
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
github.com/moby/patternmatcher v0.5.0 // indirect
Expand All @@ -97,21 +97,21 @@ require (
github.com/opencontainers/image-spec v1.1.0-rc2 // indirect
github.com/opencontainers/runc v1.1.3 // indirect
github.com/pelletier/go-toml v1.9.5 // indirect
github.com/philhofer/fwd v1.1.1 // indirect
github.com/philhofer/fwd v1.1.2 // indirect
github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rivo/uniseg v0.3.4 // indirect
github.com/rivo/uniseg v0.4.3 // indirect
github.com/rs/xid v1.4.0 // indirect
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd // indirect
github.com/sirupsen/logrus v1.9.0 // indirect
github.com/stretchr/objx v0.5.0 // indirect
github.com/tinylib/msgp v1.1.6 // indirect
github.com/tinylib/msgp v1.1.8 // indirect
github.com/vmihailenco/msgpack/v5 v5.3.5 // indirect
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
github.com/wiggin77/merror v1.0.4 // indirect
github.com/wiggin77/srslog v1.0.1 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
github.com/yuin/goldmark v1.4.13 // indirect
github.com/yuin/goldmark v1.5.3 // indirect
gitlab.com/golang-commonmark/html v0.0.0-20191124015941-a22733972181 // indirect
gitlab.com/golang-commonmark/linkify v0.0.0-20191026162114-a0c2df6c8f82 // indirect
gitlab.com/golang-commonmark/mdurl v0.0.0-20191124015652-932350d1cb84 // indirect
Expand All @@ -120,12 +120,12 @@ require (
go.opentelemetry.io/otel/metric v1.16.0 // indirect
go.opentelemetry.io/otel/trace v1.16.0 // indirect
golang.org/x/crypto v0.12.0 // indirect
golang.org/x/image v0.0.0-20220722155232-062f8c9fd539 // indirect
golang.org/x/image v0.3.0 // indirect
golang.org/x/sys v0.11.0 // indirect
golang.org/x/text v0.12.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20220817144833-d7fd3f11b9b1 // indirect
google.golang.org/grpc v1.48.0 // indirect
google.golang.org/genproto v0.0.0-20230104163317-caabf589fcbf // indirect
google.golang.org/grpc v1.51.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
Expand Down
Loading

0 comments on commit 0eb2c41

Please sign in to comment.