Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEATURE: telemetry events from devtron #517

Merged
merged 13 commits into from
Jun 23, 2021
Merged

FEATURE: telemetry events from devtron #517

merged 13 commits into from
Jun 23, 2021

Conversation

vikramdevtron
Copy link
Contributor

@vikramdevtron vikramdevtron commented Jun 14, 2021

Description

telemetry events such as Summary (app total count, user count total, env n cluster count total, ci and cd daily runs count, helm charts total count, s scan runs daily) or Heatbeat events sent to posthog.

Fixes # (issue): #407

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

  • Regular Interval, Heatbeat and summary event

Checklist:

  • The title of the PR states what changed and the related issues number (used for the release note).
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have tested it for all user roles

@vikramdevtron vikramdevtron linked an issue Jun 14, 2021 that may be closed by this pull request
@vikramdevtron vikramdevtron changed the title Feature : telemetry events from devtron FEATURE : telemetry events from devtron Jun 14, 2021
@vikramdevtron vikramdevtron self-assigned this Jun 14, 2021
client/pubsub/PosthogClient.go Outdated Show resolved Hide resolved
client/pubsub/PosthogClient.go Outdated Show resolved Hide resolved
client/pubsub/PosthogClient.go Outdated Show resolved Hide resolved
return nil, err
}

_, err = cron.AddFunc(fmt.Sprintf("@every %dm", 1), watcher.HeartbeatEventForTelemetry)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if orchestrator bounces timer will get reset

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in that case we can get more than one event for heartbeat is this will create problem ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

solved

client/telemetry/TelemetryEventClient.go Outdated Show resolved Hide resolved

func (impl *TelemetryEventClientImpl) HeartbeatEventForTelemetry() {
impl.logger.Info(">>>>>>>>>>VIKI startup heartbeat event ")
clusterBean, err := impl.clusterService.FindOne(cluster.ClusterName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cache this value in memory

client/telemetry/TelemetryEventClient.go Outdated Show resolved Hide resolved

func (impl AppListingRepositoryImpl) FindAppCount(isProd bool) (int, error) {
var count int
query := "SELECT count(distinct pipeline.app_id) from pipeline pipeline " +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use ORM query instead of native

internal/sql/repository/AppListingRepository.go Outdated Show resolved Hide resolved
internal/util/K8sUtil.go Outdated Show resolved Hide resolved
@vikramdevtron vikramdevtron changed the title FEATURE : telemetry events from devtron FEATURE: telemetry events from devtron Jun 14, 2021
cfg := &PosthogConfig{}
err := env.Parse(cfg)
if err != nil {
logger.Error("err", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

proper stack-trace will not be printed here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logger.Errorw("exception caught while parsing posthog config", "err", err)
done

ciPipelineRepository: ciPipelineRepository, pipelineRepository: pipelineRepository,
}
watcher.HeartbeatEventForTelemetry()
_, err := cron.AddFunc(fmt.Sprintf("@every %dm", 5), watcher.SummeryEventForTelemetry)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please make duration configurable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

return
}
cm, err := impl.K8sUtil.GetConfigMap(impl.aCDAuthConfig.ACDConfigMapNamespace, DevtronUniqueClientIdConfigMap, client)
if err != nil && strings.Contains(err.Error(), "not found") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have some error codes available to identify not found?
we should be using that instead of not found string match

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

return
}

client, err := impl.K8sUtil.GetClientForIncluster(cfg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicate code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

client/telemetry/TelemetryEventClient.go Show resolved Hide resolved
}
impl.PosthogClient.Client.Enqueue(posthog.Capture{
DistinctId: ucid,
Event: fmt.Sprintf("heartbeat event sent from devtron"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use short keywords as event name, filtering will be easier

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

})
}

type CronLoggerImpl struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we are using derived logger class?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not required here, removed

// creates the in-cluster config
config, err := rest.InClusterConfig()
if err != nil {
panic(err.Error())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

panic from code, is it intended ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error logged, returned error

@nishant-d nishant-d merged commit a0022aa into main Jun 23, 2021
@nishant-d nishant-d deleted the posthog-event branch June 23, 2021 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

telemetry for oss
2 participants