Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs request - discussion of concurrency model #454

Closed
the1mills opened this issue Feb 1, 2016 · 1 comment
Closed

docs request - discussion of concurrency model #454

the1mills opened this issue Feb 1, 2016 · 1 comment

Comments

@the1mills
Copy link

Hi there!

I have been thinking about using Spark for awhile now. I am mostly unfamiliar with Java based servers and I would like to know about the concurrency model used by Spark.

I have mostly been doing node.js lately, and Express is also similar to Sinatra, and I like it all.

Frankly, looking through the docs, I have no real idea how Spark achieves concurrency. Is is standard Java threads or is message passing like in Akka.io?

Is the concurrency mostly abstracted away in Spark, or do we need to worry about it?

Please let me know, thanks!. t might be useful to describe how this works in the docs (at least I didn't see it in the docs anywhere).

I see this about the available threadpool:

http://sparkjava.com/documentation.html#threadpool

my use case is that I want a small Java server that can run a certain type of CPU intensive job

I imagine I would like about 8 or so workers (about as many workers as there are cores) that can process the job.

Is there any example of how to create some sort of job queue with Spark, and have about 8 workers processing the job?

Is Spark a good match for this use case? thanks! :)

@tipsy
Copy link
Contributor

tipsy commented Mar 29, 2016

See #208

@tipsy tipsy closed this as completed Mar 29, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants