Skip to content

Commit 1c6a2ea

Browse files
weisong44weisong44
committed
Merge remote-tracking branch 'upstream/master'
2 parents a6c94ad + b8058af commit 1c6a2ea

File tree

147 files changed

+8042
-549
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

147 files changed

+8042
-549
lines changed

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,4 +30,5 @@ out/
3030
*.patch
3131
**.pyc
3232
samza-shell/src/main/visualizer/plan.json
33-
**/hs_err_pid*.log
33+
**/hs_err_pid*.log
34+
.vscode/

‎build.gradle‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,8 @@ rat {
6767
'**/bootstrap.min.css',
6868
'**/bootstrap.min.js',
6969
'**/build/**',
70+
'**/google-fonts.css',
71+
'**/ionicons.min.css',
7072
'**/font-awesome.min.css',
7173
'**/jquery-1.11.1.min.js',
7274
'**/jquery.tablesorter.min.js',

‎docs/_case-studies/TEMPLATE.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
---
2+
exclude_from_loop: true # useful for drafts
3+
layout: case-study # the layout to use
4+
hide_title: true # so we have control in case-study layout, but can still use page
5+
title: Totally awesome use-case of samza by Company # title of case study page
6+
study_domain: company.com # just the domain, not the protocol
7+
menu_title: Company # what shows up in the menu
8+
excerpt_separator: <!--more-->
9+
---
10+
<!--
11+
Licensed to the Apache Software Foundation (ASF) under one or more
12+
contributor license agreements. See the NOTICE file distributed with
13+
this work for additional information regarding copyright ownership.
14+
The ASF licenses this file to You under the Apache License, Version 2.0
15+
(the "License"); you may not use this file except in compliance with
16+
the License. You may obtain a copy of the License at
17+
18+
http://www.apache.org/licenses/LICENSE-2.0
19+
20+
Unless required by applicable law or agreed to in writing, software
21+
distributed under the License is distributed on an "AS IS" BASIS,
22+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23+
See the License for the specific language governing permissions and
24+
limitations under the License.
25+
-->
26+
27+
Testing the excerpt
28+
29+
<!--more-->
30+
31+
32+
Markdown content goes here

‎docs/_case-studies/digitalsmiths.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
layout: case-study # the layout to use
3+
hide_title: true # so we have control in case-study layout, but can still use page
4+
title: Totally awesome use-case of samza by DigitalSmiths # title of case study page
5+
study_domain: digitalsmiths.com # just the domain, not the protocol
6+
menu_title: DigitalSmiths # what shows up in the menu
7+
excerpt_separator: <!--more-->
8+
---
9+
<!--
10+
Licensed to the Apache Software Foundation (ASF) under one or more
11+
contributor license agreements. See the NOTICE file distributed with
12+
this work for additional information regarding copyright ownership.
13+
The ASF licenses this file to You under the Apache License, Version 2.0
14+
(the "License"); you may not use this file except in compliance with
15+
the License. You may obtain a copy of the License at
16+
17+
http://www.apache.org/licenses/LICENSE-2.0
18+
19+
Unless required by applicable law or agreed to in writing, software
20+
distributed under the License is distributed on an "AS IS" BASIS,
21+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+
See the License for the specific language governing permissions and
23+
limitations under the License.
24+
-->
25+
26+
Writing a brief sentence, that is different from the same blurb on the powered-by section, that prepares readers for the case study is a good idea.
27+
28+
<!--more-->
29+
30+
31+
Markdown content goes here

‎docs/_case-studies/fortscale.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
layout: case-study # the layout to use
3+
hide_title: true # so we have control in case-study layout, but can still use page
4+
title: Totally awesome use-case of samza by FortScale # title of case study page
5+
study_domain: fortscale.com # just the domain, not the protocol
6+
menu_title: FortScale # what shows up in the menu
7+
excerpt_separator: <!--more-->
8+
---
9+
<!--
10+
Licensed to the Apache Software Foundation (ASF) under one or more
11+
contributor license agreements. See the NOTICE file distributed with
12+
this work for additional information regarding copyright ownership.
13+
The ASF licenses this file to You under the Apache License, Version 2.0
14+
(the "License"); you may not use this file except in compliance with
15+
the License. You may obtain a copy of the License at
16+
17+
http://www.apache.org/licenses/LICENSE-2.0
18+
19+
Unless required by applicable law or agreed to in writing, software
20+
distributed under the License is distributed on an "AS IS" BASIS,
21+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+
See the License for the specific language governing permissions and
23+
limitations under the License.
24+
-->
25+
26+
Testing the excerpt
27+
28+
<!--more-->
29+
30+
Markdown content goes here

‎docs/_case-studies/index.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
layout: page
3+
title: Case Studies
4+
exclude_from_loop: true
5+
---
6+
<!--
7+
Licensed to the Apache Software Foundation (ASF) under one or more
8+
contributor license agreements. See the NOTICE file distributed with
9+
this work for additional information regarding copyright ownership.
10+
The ASF licenses this file to You under the Apache License, Version 2.0
11+
(the "License"); you may not use this file except in compliance with
12+
the License. You may obtain a copy of the License at
13+
14+
http://www.apache.org/licenses/LICENSE-2.0
15+
16+
Unless required by applicable law or agreed to in writing, software
17+
distributed under the License is distributed on an "AS IS" BASIS,
18+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
19+
See the License for the specific language governing permissions and
20+
limitations under the License.
21+
-->
22+
23+
Explore the many use-cases of the Samza Framework via our case-studies.
24+
25+
<ul class="case-studies">
26+
27+
{% for company in site.case-studies %}
28+
{% if company.exclude_from_loop %}
29+
{% continue %}
30+
{% endif %}
31+
<li>
32+
<a href="{{ company.url }}" title="{{ company.menu_title }}">
33+
<span style="background-image: url('https://logo.clearbit.com/{{ company.study_domain }}?size=256');"></span>
34+
</a>
35+
<div class="study-detail">
36+
<a href="https://{{ company.study_domain }}" class="external-link" rel="nofollow">
37+
<i class="icon ion-md-share-alt"></i> {{ company.menu_title }}
38+
</a>
39+
{% if company.excerpt %}
40+
<div class="study-description">
41+
{{ company.excerpt }}
42+
</div>
43+
{% endif %}
44+
<a class="btn" href="{{ company.url }}">View Case Study</a>
45+
</div>
46+
</li>
47+
{% endfor %}
48+
49+
</ul>

‎docs/_case-studies/intuit.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
layout: case-study # the layout to use
3+
hide_title: true # so we have control in case-study layout, but can still use page
4+
title: Totally awesome use-case of samza by Intuit # title of case study page
5+
study_domain: intuit.com # just the domain, not the protocol
6+
menu_title: Intuit # what shows up in the menu
7+
excerpt_separator: <!--more-->
8+
---
9+
<!--
10+
Licensed to the Apache Software Foundation (ASF) under one or more
11+
contributor license agreements. See the NOTICE file distributed with
12+
this work for additional information regarding copyright ownership.
13+
The ASF licenses this file to You under the Apache License, Version 2.0
14+
(the "License"); you may not use this file except in compliance with
15+
the License. You may obtain a copy of the License at
16+
17+
http://www.apache.org/licenses/LICENSE-2.0
18+
19+
Unless required by applicable law or agreed to in writing, software
20+
distributed under the License is distributed on an "AS IS" BASIS,
21+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+
See the License for the specific language governing permissions and
23+
limitations under the License.
24+
-->
25+
26+
Testing the excerpt
27+
28+
<!--more-->
29+
30+
31+
Markdown content goes here

‎docs/_case-studies/linkedin.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
layout: case-study # the layout to use
3+
hide_title: true # so we have control in case-study layout, but can still use page
4+
title: Totally awesome use-case of samza by LinkedIn # title of case study page
5+
study_domain: linkedin.com # just the domain, not the protocol
6+
menu_title: LinkedIn # what shows up in the menu
7+
excerpt_separator: <!--more-->
8+
---
9+
<!--
10+
Licensed to the Apache Software Foundation (ASF) under one or more
11+
contributor license agreements. See the NOTICE file distributed with
12+
this work for additional information regarding copyright ownership.
13+
The ASF licenses this file to You under the Apache License, Version 2.0
14+
(the "License"); you may not use this file except in compliance with
15+
the License. You may obtain a copy of the License at
16+
17+
http://www.apache.org/licenses/LICENSE-2.0
18+
19+
Unless required by applicable law or agreed to in writing, software
20+
distributed under the License is distributed on an "AS IS" BASIS,
21+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+
See the License for the specific language governing permissions and
23+
limitations under the License.
24+
-->
25+
26+
Testing the excerpt
27+
28+
<!--more-->
29+
30+
Markdown content goes here

‎docs/_case-studies/netflix.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
layout: case-study # the layout to use
3+
hide_title: true # so we have control in case-study layout, but can still use page
4+
title: Totally awesome use-case of samza by Netflix # title of case study page
5+
study_domain: netflix.com # just the domain, not the protocol
6+
menu_title: Netflix # what shows up in the menu
7+
excerpt_separator: <!--more-->
8+
---
9+
<!--
10+
Licensed to the Apache Software Foundation (ASF) under one or more
11+
contributor license agreements. See the NOTICE file distributed with
12+
this work for additional information regarding copyright ownership.
13+
The ASF licenses this file to You under the Apache License, Version 2.0
14+
(the "License"); you may not use this file except in compliance with
15+
the License. You may obtain a copy of the License at
16+
17+
http://www.apache.org/licenses/LICENSE-2.0
18+
19+
Unless required by applicable law or agreed to in writing, software
20+
distributed under the License is distributed on an "AS IS" BASIS,
21+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+
See the License for the specific language governing permissions and
23+
limitations under the License.
24+
-->
25+
26+
Testing the excerpt
27+
28+
<!--more-->
29+
30+
Markdown content goes here

‎docs/_case-studies/optimizely.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
layout: case-study
3+
hide_title: true # so we have control in case-study layout, but can still use page
4+
title: Real Time Session Aggregation at Optimizely
5+
study_domain: optimizely.com
6+
menu_title: Optimizely
7+
excerpt_separator: <!--more-->
8+
---
9+
<!--
10+
Licensed to the Apache Software Foundation (ASF) under one or more
11+
contributor license agreements. See the NOTICE file distributed with
12+
this work for additional information regarding copyright ownership.
13+
The ASF licenses this file to You under the Apache License, Version 2.0
14+
(the "License"); you may not use this file except in compliance with
15+
the License. You may obtain a copy of the License at
16+
17+
http://www.apache.org/licenses/LICENSE-2.0
18+
19+
Unless required by applicable law or agreed to in writing, software
20+
distributed under the License is distributed on an "AS IS" BASIS,
21+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+
See the License for the specific language governing permissions and
23+
limitations under the License.
24+
-->
25+
26+
Testing the excerpt
27+
28+
<!--more-->
29+
30+
Optimizely is a world’s leading experimentation platform, enabling businesses to deliver continuous experimentation and personalization across websites, mobile apps and connected devices. At Optimizely, billions of events are tracked on a daily basis. Session metrics are among the key metrics provided to their end user in real time. Prior to introducing Samza for realtime computation, the engineering team at Optimizely used HBase to store and serve experimentation data, and Druid for personalization data including session metrics. As business requirements evolved, the Druid-based solution became more and more challenging.
31+
32+
- Long delays in session metrics caused by M/R jobs
33+
- Reprocessing of events due to inability to incrementally update Druid index
34+
- Difficulties in scaling dimensions and cardinality
35+
- Queries expanding long time periods are expensive
36+
37+
The engineering team at Optimizely decided to move away from Druid and focus on HBase as the store, and introduced stream processing to pre-aggregate and deduplicate session events. They evaluated multiple stream processing platforms and chose Samza as their stream processing platform. In their solution, every session event is tagged with an identifier for up to 30 minutes; upon receiving a session event, the Samza job updates session metadata and aggregates counters for the session that is stored in a local RocksDB state store. At the end of each one-minute window, aggregated session metrics are ingested to HBase. With the new solution
38+
39+
- The median query latency was reduced from 40+ ms to 5 ms
40+
- Session metrics are now available in realtime
41+
- HBase query response time is improved due to reduced write-rate
42+
- HBase storage requirement are drastically reduced
43+
- Lower development effort thanks to out-of-the-box Kafka integration
44+
45+
Here is a testimonial from Optimizely
46+
47+
“At Optimizely, we have built the world’s leading experimentation platform, which ingests billions of click-stream events a day from millions of visitors for analysis. Apache Samza has been a great asset to Optimizely's Event ingestion pipeline allowing us to perform large scale, real time stream computing such as aggregations (e.g. session computations) and data enrichment on a multiple billion events / day scale. The programming model, durability and the close integration with Apache Kafka fit our needs perfectly” said Vignesh Sukumar, Senior Engineering Manager at Optimizely”
48+
49+
In addition, stream processing is also applied to other use cases such as data enrichment, event stream partitioning and metrics processing at Optimizely.
50+
51+
Key Samza features: *Stateful processing*, *Windowing*, *Kafka-integration*, *Scalability*, *Fault-tolerant*
52+
53+
More information
54+
55+
- [https://medium.com/engineers-optimizely/from-batching-to-streaming-real-time-session-metrics-using-samza-part-1-aed2051dd7a3](https://medium.com/engineers-optimizely/from-batching-to-streaming-real-time-session-metrics-using-samza-part-1-aed2051dd7a3)
56+
57+
- [https://medium.com/engineers-optimizely/from-batching-to-streaming-real-time-session-metrics-using-samza-part-2-b596350a7820](https://medium.com/engineers-optimizely/from-batching-to-streaming-real-time-session-metrics-using-samza-part-2-b596350a7820)
58+

0 commit comments

Comments
 (0)