Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NEMO-385]LambdaPass, LambdaPolicy and ResourceLambdaProperty #214

Merged
merged 6 commits into from
Jun 7, 2019

Conversation

alapha23
Copy link
Contributor

@alapha23 alapha23 commented Apr 15, 2019

JIRA: NEMO-385: Support Lambda Pass with lambda policy and lambda resource property

Major changes:

  • Implement LambdaPass, LambdaPolicy and ResourceLambdaProperty

Tests for the changes:

  • No test suites implemented

Other comments:

Test Build Command
mvn clean && mvn install -T 2C -Dmaven.test.skip=true -Dcheckstyle.skip -Dmaven.javadoc.skip=true -Drat.skip=true -pl \!compiler/frontend/spark,\!examples/spark,\!examples/beam

Execute Command

./bin/run_nexmark.sh \
        -job_id nexmark-Q0 \
        -executor_json `pwd`/examples/resources/executors/beam_test_executor_resources.json \
        -user_main org.apache.beam.sdk.nexmark.Main \
        -optimization_policy org.apache.nemo.compiler.optimizer.policy.LambdaPolicy \
        -scheduler_impl_class_name org.apache.nemo.runtime.master.scheduler.StreamingScheduler \
        -user_args "--runner=org.apache.nemo.client.beam.NemoRunner --streaming=true --query=$1 --manageResources=false --monitorJobs=true --streamTimeout=30"

@taegeonum
Copy link
Contributor

@alapha23 Could you please rebase your branch based on the current master?

wonook and others added 4 commits April 17, 2019 14:16
JIRA: [NEMO-379: Change javadoc goal to a proper one](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-379)

**Major changes:**
- Change javadoc test command from `javadoc` to `test-javadoc`. This prevents tests from creating unwanted javdoc files, caused by apache#139 .

**Minor changes to note:**
- None

**Tests for the changes:**
- None

**Other comments:**
- None

Closes apache#212

hihi
Copy link
Contributor

@taegeonum taegeonum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @alapha23 ! I've left some comments.

* Maintainer: Gao Zhiyuan<alapha23@gmail.com>.
* Resource property supporting Lambda Pass
*/
public final class ResourceLambdaProperty extends VertexExecutionProperty<ResourceLambdaProperty.Value> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't you add a LAMBDA variable in ResourcePriorityProperty class? we don't need to create another class for the resource lambda property.

Copy link
Contributor

@taegeonum taegeonum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will take a look at it again after the comments are addressed

@taegeonum
Copy link
Contributor

@johnyangk Could you please take a look at this PR?

@johnyangk
Copy link
Contributor

The PR looks good to me. 👍

@johnyangk
Copy link
Contributor

@taegeonum Can you merge? (Github won't let me do it)

@taegeonum taegeonum merged commit 5b7177b into apache:master Jun 7, 2019
alapha23 added a commit to alapha23/incubator-nemo that referenced this pull request Aug 2, 2019
…#214)

JIRA: [NEMO-385: Support Lambda Pass with lambda policy and lambda resource property](https://issues.apache.org/jira/browse/NEMO-385)

**Major changes:**
- Implement LambdaPass, LambdaPolicy and ResourceLambdaProperty

**Tests for the changes:**
- No test suites implemented

**Other comments:**
- This issue is a subissue under [NEMO-352 Nemo on AWS Lambda](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-352)
- This PR relates to GSoC2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants