Unable to retrieve messages from Beanstalk queue

[shruthi-ananth]

I am working with Beanstalk server to schedule jobs. I used Phalcon's documentation to setup a producer to put jobs onto the queue. That works alright.

I went ahead and setup a worker to watch the tube and retrieve messages to process jobs from the queue. The worker task runs from the console but does not retrieve messages from the queue. I tried it with an external client library, Pheanstalk integrated with the Phalcon app and the worker picked up the job as soon as it saw it on the tube. So it works great! I am using the 'default' tube for both implementations.

There are no error messages. But it appears that although the worker task is running Beanstalk server stats(obtained using Pheanstalk lib) does not recognize that the worker is up unlike in the case of Pheanstalk implementation. I used the following code to retrieve messages.

<?php

while (($job = $queue->peekReady()) !== false) {

    $message = $job->getBody();

    var_dump($message);

    $job->delete();
}

I would like to use Phalcon's queueing rather than loading Pheanstalk lib. Has anyone faced this issue? Please help.

[dreamsxin]

I may not correctly understand what you mean, Please submit your code to reproduce the error?
I run this code is ok

//Connect to the queue
$queue = new Phalcon\Queue\Beanstalk();

//Insert the job in the queue
$queue->put(array('processVideo' => 4871));

while (($job = $queue->peekReady()) !== false) {

    $message = $job->getBody();

    var_dump($message);

    $job->delete();
}

[shruthi-ananth]

Please find the code here: https://github.com/shruthi-ananth/phalcon-beanstalk

Once you have Beanstalk server up, you can run producer and worker tasks in the console using 'php console.php producer main' and 'php console.php worker main'. The producer puts a job on the queue and that works fine. The worker(Phalcon implementation) does not pick up a job from the queue to execute unlike the PheanstalkWorker Task.('php console.php pheanstalkworker main'). You use 'php console.php pheanstalkstats main' to check the server stats and keep both workers running to see which worker picks the job.

Also, I have the following installed on my local test environment.

Phalcon 1.2.3
PHP 5.3.1
Beanstalk Server 1.9

[jrsprice]

I try this and ->put halts everything. Nothing runs.

[ghost]

@shruthi-ananth works for me:

vladimir@sjinks:~/workspace/phalcon-beanstalk$ php console.php producer main
PHP Strict Standards:  Only variables should be assigned by reference in /home/vladimir/workspace/phalcon-beanstalk/tasks/ProducerTask.php on line 16

Strict Standards: Only variables should be assigned by reference in /home/vladimir/workspace/phalcon-beanstalk/tasks/ProducerTask.php on line 16
Connecting to beanstalk in Producer
Servers: localhost
Watching tube: default

----------------------------
Producer says: Putting a job on the queue.
4vladimir@sjinks:~/workspace/phalcon-beanstalk$ php console.php worker main
Connecting to beanstalk in Worker
Servers: localhost
Watching tube: default

----------------------------
Worker says: Wait, watch and pick up a job on the queue.
message array(3) {
  ["user_id"]=>
  int(1)
  ["category"]=>
  string(8) "delegate"
  ["message"]=>
  string(32) "Someone wants you as a delegate!"
}
^C

I had to patch WorkerTask.php a bit though:

diff --git a/tasks/WorkerTask.php b/tasks/WorkerTask.php
index ba2e2d5..a6a75ff 100755
--- a/tasks/WorkerTask.php
+++ b/tasks/WorkerTask.php
@@ -13,7 +13,7 @@ class WorkerTask extends \Phalcon\CLI\Task
     //protected $_queue = null;
     public function initialize() {
         $this->_connection = new Phalcon\Queue\Beanstalk($this->_config);
-        $this->connected =& is_resource($this->_connection);
+        $this->connected = is_resource($this->_connection);
         echo 'Connecting to beanstalk in '.$this->_name."\n";
         echo 'Servers: '.implode(',', $this->_config)."\n";
         echo 'Watching tube: '.$this->_tube."\n";
@@ -58,7 +58,8 @@ class WorkerTask extends \Phalcon\CLI\Task
             //var_dump($job);
             $message = $job->getBody();

-            var_dump("message ".$message);
+            echo "message ";
+            var_dump($message);

             $job->delete();
         }

@jrsprice do you have a sample code? i want to try to reproduce the bug

[jrsprice]

@sjinks I have the same code and it just halts on put with the producer task.

Same code etc. I have tried both beanstalkd 1.4 and beanstalkd 1.9. Which version of beanstalk?

[ghost]

Version: 1.8-2 from Ubuntu 12.10

What OS do you use? I can set it up in a virtual box and try to reproduce the issue.

[jrsprice]

I have my setup on a vm Ubuntu 12.04

I haven't been able to get phalcon + beanstalk working at all. I did a simple sudo apt-get install beanstalkd ended up w/ v 1.4

I really appreciate you looking into it as well. I've been stuck on this issue for a while. Would love to get beanstalkd running and compare it with gearman which currently works.

[jrsprice]

When I run the producer task, it seems as though the connection remains open and gets stuck on the put method. I can kill my running service and the worker will output the job details. Seems like beanstalkd not running like it should on my vm.

[ghost]

This test: https://github.com/sjinks/cphalcon/blob/beanstalk/unit-tests/BeanstalkTest.php successfully runs on Travis

[jrsprice]

Def. not working for me still. I have verbose output with my beanstalkd 1.9. When I run my tasks the worker listens while the producer halts on the put method. No error when connecting. Very odd.

[ghost]

Confirmed — does not work on 12.04.

[ghost]

Could you please modify ext/queue/beanstalk.c as follows:

diff --git a/ext/queue/beanstalk.c b/ext/queue/beanstalk.c
index 90d2ec6..03b3370 100644
--- a/ext/queue/beanstalk.c
+++ b/ext/queue/beanstalk.c
@@ -492,7 +492,7 @@ PHP_METHOD(Phalcon_Queue_Beanstalk, read){
                ZVAL_STRING(end_of_file, "\r\n", 1);

                PHALCON_INIT_NVAR(packet);
-               phalcon_call_func_p3(packet, "stream_get_line", connection, total_length, end_of_file);
+               phalcon_call_func_p2(packet, "fgets", connection, total_length);
        }

        RETURN_CCTOR(packet);

then recompile Phalcon from ext/:

cd ext
phpize
./configure
make
sudo make install

and check if this solves the issue?

[jrsprice]

That works! Thanks a bunch.

[ghost]

Fixed in #1619 and #1621, can be closed now.

[zikezhang]

actually, Phalcon can't delete beanstalk job.

[sergeyklay]

@zikezhang Could you open a separated issue with code to reproduce?