Skip to content

Conversation

eranra
Copy link
Collaborator

@eranra eranra commented Jan 23, 2022

No description provided.

@codecov-commenter
Copy link

Codecov Report

❗ No coverage uploaded for pull request base (main@e4e2316). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main      #45   +/-   ##
=======================================
  Coverage        ?   42.80%           
=======================================
  Files           ?       25           
  Lines           ?     1203           
  Branches        ?        0           
=======================================
  Hits            ?      515           
  Misses          ?      659           
  Partials        ?       29           
Flag Coverage Δ
unittests 42.80% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e4e2316...34b64d6. Read the comment docs.

@@ -20,6 +20,7 @@ FL2M_BIN_FILE=flowlogs2metrics
CG_BIN_FILE=confgenerator
NETFLOW_GENERATOR=nflow-generator
CMD_DIR=./cmd/
FL2M_CONF_FILE ?= contrib/kubernetes/flowlogs2metrics.conf.yaml
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using this FL2M_CONF_FILE as the name for the configuration file: This is used twice in the makefile + this allows us to deploy with different file than the default.

@@ -58,7 +59,7 @@ lint: $(GOLANGCI_LINT) ## Lint the code
.PHONY: build_code
build_code: validate_go lint
@go mod vendor
go build "${CMD_DIR}${FL2M_BIN_FILE}"
VERSION=$$(date); go build -ldflags "-X 'main.Version=$$VERSION'" "${CMD_DIR}${FL2M_BIN_FILE}"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the data as the Version variable in the main package ... we use that to report version when we start flowlogs 2metrics

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a version number to flowlogs2 metrics is a-bit beyond the title of this PR but I really needed that because I changed the code all the time, and the version (in this case the compile time ) is very useful to know what image you are using old or now, etc,.

configAsJSON, _ := json.MarshalIndent(config.Opt, "", "\t")
log.Infof("configuration:\n%s\n", configAsJSON)
configAsJSON, _ := json.MarshalIndent(config.Opt, "", " ")
fmt.Printf("Using configuration:\n%s\n", configAsJSON)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this to be printf so we can always get this even if debug is in error (very useful for Palmetto)

@@ -155,7 +156,7 @@ func run() {
)

// Starting log message
log.Infof("starting %s", filepath.Base(os.Args[0]))
fmt.Printf("%s starting - version [%s]\n\n", filepath.Base(os.Args[0]), Version)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reporting version number on start up (this is seen in the container when we run flowlogs2metrics)
Moving this to be printf so we can always get this even if debug is in error (very useful for Palmetto)

@@ -13,13 +13,14 @@ COPY cmd/ cmd/
COPY pkg/ pkg/

# Build
RUN CGO_ENABLED=0 GOOS=linux GO111MODULE=on GOARCH=amd64 go build ./cmd/flowlogs2metrics
RUN VERSION=$(date); CGO_ENABLED=0 GOOS=linux GO111MODULE=on GOARCH=amd64 go build -ldflags "-X 'main.Version=$VERSION'" ./cmd/flowlogs2metrics
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding version when we compile the docker image (same as when we compile manually in the makefile)

@@ -173,7 +173,7 @@ pipeline:
- srcK8S_Namespace
- srcK8S_Type
operation: sum
recordkey: Bytes
recordkey: bytes
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small bug -- using need to use bytes and not Bytes we convert this in the generic transform

@@ -1,5 +1,5 @@
# This file was generated automatically by flowlogs2metrics confgenerator
log-level: debug
log-level: error
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel that we are good to work in error level now ... :-)

echo "Use: kubectl logs -l app=flowlogs2metrics | grep pod-to-pod-workload"
}

main
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the main file of this PR ... it adds a test that deploys ingress, egress, and pod-to-pod in OCP and then deploys flowlogs2metrics to be able to see the flow-logs ... this is done without any aggregation. Just the pure raw flow logs (after we add k8s info to the logs)

@@ -25,7 +25,7 @@ import (

func (cg *ConfGen) generateFlowlogs2MetricsConfig(fileName string) error {
config := map[string]interface{}{
"log-level": "debug", // TODO: remove this temporal debug statement
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, ready for error level :-)

@@ -28,7 +30,7 @@ type writeStdout struct {
func (t *writeStdout) Write(in []interface{}) {
log.Debugf("entering writeStdout Write")
for _, v := range in {
log.Infof("%v", v)
fmt.Printf("%s: %v\n", time.Now().Format(time.StampMilli), v)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we work with stdout it is not connected to log level we just want to print the output to stdout

@eranra eranra merged commit 2bca692 into netobserv:main Jan 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants