Skip to content

Conversation

@joanhey
Copy link
Contributor

@joanhey joanhey commented Jan 31, 2026

All the removed permutations are the same, only with 1 test. The runs need more time without any benefit.

Example with fortunes:
image

With plaintext:
image

The cache_query was also removed, as fail in all runs:
https://tfb-status.techempower.com/unzip/results.2026-01-26-17-36-11-536.zip/results/20260118211352/water-http-cached/cached-query/verification.txt
And need to be in the default test, but also fail.

Change Dockerfile, as is not permitted to use latest version.

@HassanSharara

@HassanSharara
Copy link
Contributor

can i have more info about cached-queries cause its 100% worked on my side
and also the error message

VERIFYING CACHED-QUERY

FAIL for http://10.0.1.1:8080
Server did not respond to request
See https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview#specific-test-requirements

 do i need to provide http response for / path ?
 cause in source code 
 ```rust 
 WaterController! {
holder -> super::MainType,
shared -> super::SH,
name -> EntryController,
functions -> {

    GET -> "cached-queries" -> query (context) {
        _=super::handle_cached_queries(context).await;
    }
}

}
```

 i only provided response for /cached-queries?q= path 

@HassanSharara
Copy link
Contributor

{
"uuid": "3bf42fcc-8b97-4ec9-ab0f-efb87e5d988c",
"name": "(unspecified, datetime = 2026-01-31 10:57:19)",
"environmentDescription": "(unspecified, hostname = 0737d79c0ec5)",
"git": {
"commitId": "ac160124ea293a18aabca08cf1e9821d496184f8",
"repositoryUrl": "https://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@github.com/HassanSharara/water_http-techempower",
"branchName": "master"
},
"startTime": 1769857040210,
"completionTime": 1769857727425,
"concurrencyLevels": [
16,
32,
64,
128,
256,
512
],
"pipelineConcurrencyLevels": [
256,
1024,
4096,
16384
],
"queryIntervals": [
1,
5,
10,
15,
20
],
"cachedQueryIntervals": [
1,
10,
20,
50,
100
],
"frameworks": [
"water-http-cached"
],
"duration": 15,
"rawData": {
"cached-query": {
"water-http-cached": [
{
"latencyAvg": "1.22ms",
"latencyStdev": "1.94ms",
"latencyMax": "31.16ms",
"totalRequests": 13039171,
"connect": 0,
"read": 0,
"write": 0,
"timeout": 439,
"startTime": 1769857585,
"endTime": 1769857601
},
{
"latencyAvg": "1.26ms",
"latencyStdev": "1.92ms",
"latencyMax": "34.94ms",
"totalRequests": 11969012,
"startTime": 1769857603,
"endTime": 1769857618
},
{
"latencyAvg": "1.34ms",
"latencyStdev": "2.01ms",
"latencyMax": "31.98ms",
"totalRequests": 11413827,
"connect": 0,
"read": 0,
"write": 0,
"timeout": 151,
"startTime": 1769857620,
"endTime": 1769857635
},
{
"latencyAvg": "1.41ms",
"latencyStdev": "1.76ms",
"latencyMax": "41.85ms",
"totalRequests": 7923820,
"startTime": 1769857637,
"endTime": 1769857652
},
{
"latencyAvg": "1.49ms",
"latencyStdev": "1.61ms",
"latencyMax": "23.86ms",
"totalRequests": 6752236,
"connect": 0,
"read": 0,
"write": 0,
"timeout": 288,
"startTime": 1769857654,
"endTime": 1769857669
}
]
},
}

@joanhey
Copy link
Contributor Author

joanhey commented Jan 31, 2026

In my local test, also work the cached queries. But not in Citrine servers.
Perhaps need more time to cache the queries, as is creating 56 workers.

EDIT: and remember to add it in the default test.

@HassanSharara
Copy link
Contributor

i need to simulate these servers to see the exact issue
and i dont think its time issue i think its config issue according to the message


VERIFYING CACHED-QUERY

FAIL for http://10.0.1.1:8080
Server did not respond to request
See https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview#specific-test-requirements

@joanhey
Copy link
Contributor Author

joanhey commented Jan 31, 2026

Also have more fails in the bench and tests.

https://tfb-status.techempower.com/results/a12d8f43-58bc-4ad1-9277-aa5ab294c10b
image

https://tfb-status.techempower.com/results/acc1ad82-ae2c-4d1d-a600-c7ff9d0c5917
image

PD: the only difference is the number of CPUs (and workers in the app) using the Citrine servers. As pass the tests in local and in the github actions, than use less CPUs.

@HassanSharara
Copy link
Contributor

Is there full Log that could detect the real issue

@joanhey
Copy link
Contributor Author

joanhey commented Jan 31, 2026

Any log will only detect not response.

The problem is in the app, and the app need to create the log (not in a file) from what happens.

Logs:
https://tfb-status.techempower.com/unzip/results.2026-01-26-17-36-11-536.zip/water-http

@HassanSharara
Copy link
Contributor

So the app is not responding to any request of cached queries and update queries
Or it is responding at first, and then it's failed

@joanhey
Copy link
Contributor Author

joanhey commented Jan 31, 2026

The first request is with the test, if the test fail for that url, stop the process to run the bench for that url.

@joanhey
Copy link
Contributor Author

joanhey commented Jan 31, 2026

Also test it in local, but no the tests only, but the bench.
Perhaps you find the problem.

EDIT: ./tfb --mode benchmark --test water-http

@HassanSharara
Copy link
Contributor

i think its matter of host or port configurations or i dont know cause i tried to increase the workers to 56 and its still working correctly
and i already did mode benchmark cause i never trust mode test

@joanhey
Copy link
Contributor Author

joanhey commented Jan 31, 2026

The problem is in the app, like I said before.
And you need to find the problem !!!

More than 800 frameworks permutations work OK with the Citrine servers.

@HassanSharara
Copy link
Contributor

ok i am working on it
and may i know what is the used exact command to run benchmark on citrine server cause i have real one and imma bench it right now

@joanhey
Copy link
Contributor Author

joanhey commented Jan 31, 2026

In your case is a not response, in other frameworks is a time out problem.

It's your app that need to log the problems. Example with perl/kelp showing the problem:
https://tfb-status.techempower.com/unzip/results.2026-01-26-17-36-11-536.zip/results/20260118211352/kelp-twiggy-mysql/run/kelp-twiggy-mysql.log

@joanhey
Copy link
Contributor Author

joanhey commented Jan 31, 2026

An Http server is a black box, without knowledge of how the server processes them internally.
We send a request and return a response (if all it's working).
The only way to find the problems, is log what it is happening in the server.

@HassanSharara
Copy link
Contributor

i finished everything and iam uploading new version my repo but also i could not add cached queries to the default test cause
it`s not implement same connection and shared stuct
is that ok ?

@joanhey
Copy link
Contributor Author

joanhey commented Feb 1, 2026

For me is NOT OK to need another test only for the cached queries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants