Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gergo/automateBackendModule #2202

Merged
merged 105 commits into from
May 17, 2024
Merged
Show file tree
Hide file tree
Changes from 72 commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
3603363
feat: automation run trigger logic
gjedlicska Apr 2, 2024
14b6a04
feat: automate module, automation creation and trigger
gjedlicska Apr 10, 2024
8422eca
Merge branch 'fabians/fe2-automate-integration' of github.com:speckle…
gjedlicska Apr 10, 2024
9b2fde9
test: fix automate module tests
gjedlicska Apr 10, 2024
ea175cd
test: fixt automate module tests
gjedlicska Apr 10, 2024
d1d09e3
Merge branch 'gergo/automateBackendModule' of github.com:specklesyste…
gjedlicska Apr 10, 2024
a10830e
Merge branch 'fabians/fe2-automate-integration' into gergo/automateBa…
fabis94 Apr 22, 2024
26aa9ba
minor fix ups
fabis94 Apr 22, 2024
d7d3a46
fixed up existing backend stuff
fabis94 Apr 23, 2024
a5e0687
fixing eslint hopefully
fabis94 Apr 23, 2024
fafe3b7
tryna fix eslint
fabis94 Apr 23, 2024
abb77f7
automate code validation mechanism added
fabis94 Apr 23, 2024
3a99628
minor GQL schema change
fabis94 Apr 23, 2024
3ba0d3d
maybethis fixes eslint?
fabis94 Apr 23, 2024
f4f0230
more eslint debugging
fabis94 Apr 23, 2024
946ed82
fix cross-env missing
fabis94 Apr 23, 2024
306acf8
tryna fix eslint memory issues
fabis94 Apr 23, 2024
18d2791
ci test fix
fabis94 Apr 23, 2024
a796909
error improvements
fabis94 Apr 24, 2024
2b34120
migrations for fn tables
fabis94 Apr 24, 2024
ed03223
WIP function create
fabis94 Apr 24, 2024
ee7d83c
fn creation tests added
fabis94 Apr 25, 2024
c3cfb8f
Merge branch 'fabians/fe2-automate-integration' into gergo/automateBa…
fabis94 Apr 25, 2024
f2b35a5
fn update w/ tests
fabis94 Apr 26, 2024
bd7b0ee
fn release creation w/ tests
fabis94 Apr 26, 2024
aa5b617
fixing tests
fabis94 Apr 26, 2024
4408fd9
GH auth endpoint
fabis94 Apr 26, 2024
cae69ed
minor cleanup
fabis94 Apr 26, 2024
b7ba706
WIP reporting function statuses
fabis94 Apr 29, 2024
cbf0349
automation update/delete w/ tests
fabis94 Apr 29, 2024
d4dbd9d
WIP automation revision & trigger tests
fabis94 Apr 29, 2024
3ba75c7
revision creation tests done
fabis94 Apr 30, 2024
082e975
trigger tests
fabis94 Apr 30, 2024
124c716
function run reporting works
fabis94 Apr 30, 2024
8c03835
report status tests WIP
fabis94 Apr 30, 2024
b435cab
run status update tests done
fabis94 Apr 30, 2024
cc498cf
auth code handshake tests
fabis94 Apr 30, 2024
25bc76b
a couple of FE2 fixes
fabis94 Apr 30, 2024
18599e6
WIP function retrieval queries & tests
fabis94 Apr 30, 2024
03c378f
WIP automation queries
fabis94 May 2, 2024
92db9eb
removed all functions stuff
fabis94 May 3, 2024
b87873f
implemented fn queries
fabis94 May 3, 2024
17fae3b
all kinds of queries & resolvers done
fabis94 May 3, 2024
3e06c61
more queries
fabis94 May 3, 2024
3bff26e
automations query
fabis94 May 3, 2024
34a9658
automation status resolution core algo
fabis94 May 6, 2024
00ba99f
FE2 fixes
fabis94 May 6, 2024
2c5908f
fixed up mocks
fabis94 May 6, 2024
fb182ab
fix(fe2): disallow loading automations if non-owner
fabis94 May 6, 2024
0230555
Merge branch 'fabians/fe2-automate-integration' into gergo/automateBa…
fabis94 May 6, 2024
caf838c
fixing some benjamins changes
fabis94 May 6, 2024
12402af
hydration mismatch fix
fabis94 May 6, 2024
6cae992
fixed tests
fabis94 May 6, 2024
c8afcef
preview service fix?
fabis94 May 6, 2024
76d757b
env flag fix
fabis94 May 6, 2024
3fef121
more form validation improvements
fabis94 May 7, 2024
c7b2b45
proper automation status run ordering
fabis94 May 7, 2024
d0419bb
featured mock fix
fabis94 May 7, 2024
3c91af0
feat(fe2): promo banner support (#2242)
andrewwallacespeckle May 7, 2024
92eed81
meta data fixed
fabis94 May 7, 2024
b2d973f
introduce outdated label
fabis94 May 7, 2024
cc76b32
log streaming mock moved to serverside
fabis94 May 7, 2024
ec78d31
Viewer API Improvements (#2072)
AlexandruPopovici May 7, 2024
cff6f4f
encryption in create for FE
fabis94 May 7, 2024
f46e314
fix: integration work
gjedlicska May 7, 2024
3ecf0a3
bugfix(fe2): Fix conditioning around posting comments in viewer (#2246)
andrewwallacespeckle May 8, 2024
a833da2
core encryption stuff implemented
fabis94 May 8, 2024
4476a0a
Merge branch 'gergo/automateBackendModule' of github.com:specklesyste…
fabis94 May 8, 2024
a9ea6ac
fixing tests & linting
fabis94 May 8, 2024
de2c8ce
improved revision input validation
fabis94 May 8, 2024
b32c56f
automation create works
fabis94 May 9, 2024
e9ee53b
fix(server): avoid removing verified email from user (#2249)
alemagio May 9, 2024
8d8d7b3
automations status fix
fabis94 May 9, 2024
39926aa
fixed automation run queries
fabis94 May 9, 2024
b3be319
minor cleanup
fabis94 May 9, 2024
70858b3
implemented log streaming
fabis94 May 9, 2024
abae7b1
properly handing redacted props in update rev flow
fabis94 May 10, 2024
2010543
implemented subscriptions backend
fabis94 May 10, 2024
e72b193
feat(fe2): move settings to tab on projects page (#2207)
andrewwallacespeckle May 10, 2024
213246a
WIP subscriptions FE implementation
fabis94 May 10, 2024
4378dd2
Use showbutton prop instead of useCheckViewerCommentingAccess (#2251)
andrewwallacespeckle May 10, 2024
3e0b154
subscriptions work?
fabis94 May 10, 2024
9bb101f
Reverted project access copy to something like before (#2254)
benjaminvo May 13, 2024
3a5882f
bugfix(ui-components): Scroll jumps to top on Dialog open (#2250)
andrewwallacespeckle May 14, 2024
1ad5e01
Use shortened header in Scene Explorer (#2256)
andrewwallacespeckle May 15, 2024
a6ead06
feat(fe2) - Replace CommonEditableTitleDescription with new CommonTit…
andrewwallacespeckle May 15, 2024
7a1d4ba
Only show manage button when user canEdit (#2260)
andrewwallacespeckle May 15, 2024
3abbf17
feat: add docker compose based reverse proxy for the server stack
gjedlicska May 15, 2024
ab6938a
revert: restore docker compose ingress dockerfile
gjedlicska May 16, 2024
eff368b
chore: disable automate module feature flag by default
gjedlicska May 16, 2024
abea78a
fix: move nginx ingress file to the right place
gjedlicska May 16, 2024
69decaf
feat(fe2): Hide settings tab for logged out users (#2261)
andrewwallacespeckle May 16, 2024
d9ac23f
bugfix(fe2): Coloring function disappears if parameter title is too l…
andrewwallacespeckle May 16, 2024
29bd264
Merge branches 'gergo/automateBackendModule' and 'main' of github.com…
gjedlicska May 16, 2024
1c2827a
Implement `automateFunctionRunStatusReport` (#2262)
cdriesler May 16, 2024
16f42f6
fix: make tab selection robust by using dynamic ID lookup
andrewwallacespeckle May 16, 2024
6448040
tests: fix authz module tests
gjedlicska May 17, 2024
ff64975
fix: frontend TS issue
gjedlicska May 17, 2024
5aa4f04
ci: add automate encryption keys path value
gjedlicska May 17, 2024
116f6b6
Fix ts build errors from ui-components changes
andrewwallacespeckle May 17, 2024
9b19546
fix: frontend automation status colors
gjedlicska May 17, 2024
ef5b9ef
Merge branch 'gergo/automateBackendModule' of github.com:specklesyste…
gjedlicska May 17, 2024
513d16b
add handling for all enum cases in useRunStatusMetadata
andrewwallacespeckle May 17, 2024
8fb3dc5
Merge branch 'gergo/automateBackendModule' of https://github.com/spec…
andrewwallacespeckle May 17, 2024
819e94a
Fix merge issue
andrewwallacespeckle May 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,7 @@ jobs:
type: string
docker:
- image: speckle/pre-commit-runner:latest
resource_class: medium
resource_class: large
working_directory: *work-dir
steps:
- checkout
Expand Down Expand Up @@ -416,6 +416,7 @@ jobs:
S3_CREATE_BUCKET: 'true'
REDIS_URL: 'redis://127.0.0.1:6379'
S3_REGION: '' # optional, defaults to 'us-east-1'
FF_AUTOMATE_MODULE_ENABLED: true
steps:
- checkout
- restore_cache:
Expand Down
3 changes: 3 additions & 0 deletions .husky/pre-commit
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ if [ -n "$CI" ]
then
echo "running eslint"
yarn lint
echo "...eslint done"
echo "running prettier"
yarn prettier:check
echo "...prettier done"
else
# shellcheck disable=SC1090
. "$(dirname "$0")/_/husky.sh"
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"build:public": "yarn workspaces foreach -ptv --no-private run build",
"build:tailwind-deps": "yarn workspaces foreach -iv -j unlimited --include '{@speckle/shared,@speckle/tailwind-theme,@speckle/ui-components}' run build",
"ensure:tailwind-deps": "node ./utils/ensure-tailwind-deps.mjs",
"lint": "eslint . --ext .js,.ts,.vue --max-warnings=0",
"lint": "node --max-old-space-size=4096 ./node_modules/eslint/bin/eslint.js . --ext .js,.ts,.vue --max-warnings=0",
"helm:readme:generate": "./utils/helm/update-schema-json.sh",
"prettier:check": "prettier --check .",
"prettier:fix": "prettier --write .",
Expand Down Expand Up @@ -45,6 +45,7 @@
"@types/eslint": "^8.4.1",
"@types/lockfile": "^1.0.2",
"commitizen": "^4.2.5",
"cross-env": "^7.0.3",
"cz-conventional-changelog": "^3.3.0",
"eslint": "^8.11.0",
"eslint-config-prettier": "^8.5.0",
Expand Down
79 changes: 62 additions & 17 deletions packages/frontend-2/components/automate/automation/CreateDialog.vue
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
v-else-if="
enumStep === AutomationCreateSteps.FunctionParameters && selectedFunction
"
ref="parametersStep"
v-model:parameters="functionParameters"
v-model:has-errors="hasParameterErrors"
:fn="selectedFunction"
Expand Down Expand Up @@ -69,10 +70,15 @@ import {
import { useForm } from 'vee-validate'
import {
useCreateAutomation,
useCreateAutomationRevision
useCreateAutomationRevision,
useUpdateAutomation
} from '~/lib/projects/composables/automationManagement'
import { formatJsonFormSchemaInputs } from '~/lib/automate/helpers/jsonSchema'
import { projectAutomationRoute } from '~/lib/common/helpers/route'
import {
useAutomationInputEncryptor,
type AutomationInputEncryptor
} from '~/lib/automate/composables/automations'

enum AutomationCreateSteps {
SelectFunction,
Expand Down Expand Up @@ -124,7 +130,9 @@ const stepsWidgetData = computed(() => [
}
])

const inputEncryption = useAutomationInputEncryptor({ ensureWhen: open })
const logger = useLogger()
const updateAutomation = useUpdateAutomation()
const createAutomation = useCreateAutomation()
const createRevision = useCreateAutomationRevision()
const { enumStep, step } = useEnumSteps({ order: stepsOrder })
Expand All @@ -134,6 +142,8 @@ const {
shouldShowWidget: shouldShowStepsWidget
} = useEnumStepsWidgetSetup({ enumStep, widgetStepsMap: stepsWidgetData })

const parametersStep = ref<{ submit: () => Promise<void> }>()

const creationLoading = ref(false)
const automationId = ref<string>()
const automationName = ref<string>()
Expand All @@ -148,6 +158,7 @@ const buttons = computed((): LayoutDialogButton[] => {
case AutomationCreateSteps.SelectFunction:
return [
{
id: 'selectFnNext',
text: 'Next',
props: {
iconRight: ChevronRightIcon,
Expand All @@ -161,6 +172,7 @@ const buttons = computed((): LayoutDialogButton[] => {
case AutomationCreateSteps.FunctionParameters:
return [
{
id: 'fnParamsPrev',
text: 'Previous',
props: {
color: 'secondary',
Expand All @@ -170,17 +182,19 @@ const buttons = computed((): LayoutDialogButton[] => {
onClick: () => step.value--
},
{
id: 'fnParamsNext',
text: 'Next',
props: {
iconRight: ChevronRightIcon,
disabled: hasParameterErrors.value
},
onClick: () => step.value++
submit: true
}
]
case AutomationCreateSteps.AutomationDetails:
return [
{
id: 'detailsPrev',
text: 'Previous',
props: {
color: 'secondary',
Expand All @@ -190,6 +204,7 @@ const buttons = computed((): LayoutDialogButton[] => {
onClick: () => step.value--
},
{
id: 'detailsCreate',
text: 'Create',
submit: true,
disabled: creationLoading.value
Expand All @@ -198,6 +213,7 @@ const buttons = computed((): LayoutDialogButton[] => {
case AutomationCreateSteps.Done:
return [
{
id: 'doneClose',
text: 'Close',
props: {
color: 'secondary',
Expand All @@ -206,6 +222,7 @@ const buttons = computed((): LayoutDialogButton[] => {
onClick: () => (open.value = false)
},
{
id: 'doneGoToAutomation',
text: 'Go to Automation',
props: {
iconRight: ArrowRightIcon,
Expand Down Expand Up @@ -273,6 +290,8 @@ const onDetailsSubmit = handleDetailsSubmit(async () => {
}

creationLoading.value = true
let aId: Optional<string> = undefined
let automationEncrypt: Optional<AutomationInputEncryptor> = undefined
try {
const createRes = await createAutomation({
projectId: project.id,
Expand All @@ -281,29 +300,35 @@ const onDetailsSubmit = handleDetailsSubmit(async () => {
enabled: false
}
})
const aId = (automationId.value = createRes?.id)
aId = automationId.value = createRes?.id
if (!aId) {
logger.error('Failed to create automation', { createRes })
return
}

const parametersString = parameters
? JSON.stringify(
formatJsonFormSchemaInputs(parameters, fnRelease.inputSchema, {
clone: true
})
)
: null
automationEncrypt = await inputEncryption.forAutomation({
automationId: aId,
projectId: project.id
})

const cleanParams =
formatJsonFormSchemaInputs(parameters, fnRelease.inputSchema, {
clone: true
}) || null
const encryptedParams = automationEncrypt.encryptInputs({
inputs: cleanParams
})

const revisionRes = await createRevision(
{
projectId: project.id,
input: {
automationId: aId,
functions: [
{
functionReleaseId: fnRelease.id,
functionId: fn.id,
releaseId: fnRelease.id,
parameters: parametersString
parameters: encryptedParams
}
],
triggerDefinitions: <Automate.AutomateTypes.TriggerDefinitionsSchema>{
Expand All @@ -319,17 +344,37 @@ const onDetailsSubmit = handleDetailsSubmit(async () => {
},
{ hideSuccessToast: true }
)
if (revisionRes?.id) {
step.value++

if (!revisionRes?.id) {
logger.error('Failed to create revision', { revisionRes })
return
}

// Enable
await updateAutomation({
projectId: project.id,
input: {
id: aId,
enabled: true
}
})

step.value++
} finally {
creationLoading.value = false
automationEncrypt?.dispose()
}
})

const onDialogSubmit = (e: SubmitEvent) => {
if (enumStep.value !== AutomationCreateSteps.AutomationDetails) return
onDetailsSubmit(e)
const onDialogSubmit = async (e: SubmitEvent) => {
if (enumStep.value === AutomationCreateSteps.AutomationDetails) {
await onDetailsSubmit(e)
} else if (enumStep.value === AutomationCreateSteps.FunctionParameters) {
await parametersStep.value?.submit()
if (!hasParameterErrors.value) {
step.value++
}
}
}

watch(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
<div>
<template v-if="finalParams">
<FormJsonForm
ref="jsonForm"
v-model:data="parameters"
:schema="finalParams"
class="space-y-4"
:validate-on-mount="false"
@change="handler"
/>
</template>
Expand All @@ -16,6 +18,7 @@
</div>
</template>
<script setup lang="ts">
import type { JsonFormsChangeEvent } from '@jsonforms/vue'
import type { Optional } from '@speckle/shared'
import { useJsonFormsChangeHandler } from '~/lib/automate/composables/jsonSchema'
import { formatVersionParams } from '~/lib/automate/helpers/jsonSchema'
Expand All @@ -38,6 +41,8 @@ const props = defineProps<{
fn: AutomateAutomationCreateDialogFunctionParametersStep_AutomateFunctionFragment
}>()

const jsonForm = ref<{ triggerChange: () => Promise<Optional<JsonFormsChangeEvent>> }>()

const finalParams = computed(() => formatVersionParams(release.value?.inputSchema))
const { handler, hasErrors: hasJsonFormErrors } = useJsonFormsChangeHandler({
schema: finalParams
Expand All @@ -52,6 +57,10 @@ const release = computed(() =>
props.fn.releases.items.length ? props.fn.releases.items[0] : undefined
)

const submit = async () => {
await jsonForm.value?.triggerChange()
}

// watch(
// release,
// () => {
Expand All @@ -69,4 +78,6 @@ watch(
},
{ immediate: true }
)

defineExpose({ submit })
</script>
24 changes: 18 additions & 6 deletions packages/frontend-2/components/automate/function/Card.vue
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,12 @@
{{ fn.name }}
</RouterLink>
</div>
<div class="label-light">by {{ fn.creator?.name || 'Deleted User' }}</div>
<div class="label-light flex items-center space-x-1">
<span>by</span>
<CommonTextLink external :to="fn.repo.url" size="sm">
{{ fn.repo.owner }}
</CommonTextLink>
</div>
</div>
</div>
<div class="label-light text-foreground-2 line-clamp-3 h-16">
Expand Down Expand Up @@ -52,10 +57,14 @@
<div class="absolute top-0 right-0">
<div
v-if="hasLabel"
class="rounded-bl-lg rounded-tr-[7px] text-xs px-2 py-1 text-foreground"
:class="{ 'bg-foundation-focus': fn.isFeatured }"
class="rounded-bl-lg rounded-tr-[7px] text-xs px-2 py-1"
:class="{
'bg-foundation-focus text-foreground': fn.isFeatured,
'bg-warning text-foreground-on-primary': isOutdated
}"
>
<template v-if="fn.isFeatured">Featured</template>
<template v-if="isOutdated">Outdated</template>
<template v-else-if="fn.isFeatured">Featured</template>
</div>
</div>
</div>
Expand All @@ -75,8 +84,10 @@ graphql(`
isFeatured
description
logo
creator {
repo {
id
url
owner
name
}
}
Expand All @@ -93,10 +104,11 @@ const props = defineProps<{
noButtons?: boolean
externalMoreInfo?: boolean
selected?: boolean
isOutdated?: boolean
}>()

const NuxtLink = resolveComponent('NuxtLink')
const hasLabel = computed(() => props.fn.isFeatured)
const hasLabel = computed(() => props.fn.isFeatured || props.isOutdated)
const { html: plaintextDescription } = useMarkdown(
computed(() => props.fn.description || ''),
{ plaintext: true }
Expand Down
Loading