Data correction documentary #26

astellhorn · 2024-01-12T12:41:03Z

Documentary for functions and dependencies.
Some notes are added that can be deleted later.
Changes to previously discussed issues: The empty-glass-transmission $T^{g,p|a}$ of polarizer/analyzer we previously assumed as constant. That is not precisely true (though never any wavelength dependence was able to be measured on 3He-cells so far). To keep a wavelength-dependence as an option, the new proposition is to insert it as wavelength-dependent function $T^{g,p|a}(\lambda)$, instead of a scalar. This function will be pre-determined during cell-fabrication and hence this will not change anything in the workflow, except for reading in a pre-known function of $T^{g,p|a}$ instead of the scalar variable.

Documentary for functions and dependencies

SimonHeybrock

Not a real review, just brief comments about formatting (problems and readability).

SimonHeybrock · 2024-01-12T13:08:16Z

docs/Data correction documentary-cleared.ipynb

Use lowercase and now spaces in filenames.

True!! I am very sorry, I need to get rid of this windows habit

docs/Data correction documentary-cleared.ipynb

SimonHeybrock · 2024-01-12T13:14:19Z

docs/Data correction documentary-cleared.ipynb

+    "|  $l^{p|a}$   |  length of $^3$He-polarizer/analyzer   |  scalar, 1-dim   | from metadata   |Row 1   |\n",
+    "|  $T^{g, p|a}(\\lambda)$   |  transmission of empty glass of polarizer/analyzer   |  scalar, 1-dim   | from metadata  |Row 1   |\n",
+    "|  $O^{p|a}(\\lambda)=O^{0, p|a}*\\lambda$   |  wavelength-dependent opacity of $^3$He-polarizer/analyzer  |  continuous function of $\\lambda$   | $O^{0, p|a}$ fit to data |Row 1   |\n",
+    "|  $O^{0, p|a}$   |  wavelength-independent pre-factor of opacity of $^3$He-polarizer/analyzer   |  scalar, 1-dim   | fit to data  |Row 1   |\n",


The He3 also is not rendered.

thanks, its also not in the rest of the text I just see!

docs/Data correction documentary-cleared.ipynb

SimonHeybrock · 2024-01-12T13:24:30Z

docs/Data correction documentary-cleared.ipynb

+    "\n",
+    "| Value | Meaning | shape | How acquired? | How named in current main branch? |\n",
+    "|:--------:|:--------:|:--------:|:--------:|:--------:|\n",
+    "|  $I^{++}_{t,\\lambda}$   |  uncorrected sample data of ++ channel |  data array with columns for $I$, $\\lambda$, $t$   |  neutron data |Row 1   |\n",


I don't think the notation like $I^{++}_{t,\\lambda}$ makes sense when talking about event data. This looks like a 2-D array (for any given $t$ there can be multiple $\lambda$ and vice versa), which would be fine if we planned to histogram, but maybe need to come up with a better notation for event data? Maybe an event index?

On the other hand, maybe we can keep it, and just clarify in the beginning that the notation is for histogrammed data?

Are there any other workflows (e.g. unpol SANS or other instruments) who use a notation for event data already? I will ask around here.
But other question: the test-data we create binned, not histogrammed, but the data we use for calculating O and P(t) were histogrammed in the direct-beam data in the function "compute_direct_beam"

So for that part we use histogrammed, but for sample-correction in the end the not histogrammed data?

Let use keep histogrammed notation everywhere (at least for now). One can always think of event data as an extremely sparse histogram, so conceptually it is ok (and this document is not about the way Scipp handles event data).

docs/Data correction documentary-cleared.ipynb

SimonHeybrock · 2024-01-12T13:35:10Z

docs/Data correction documentary-cleared.ipynb

+    "\n",
+    "From the neutron measurement, this again can be retrieved by measuring the transmission through each cell (i.e., for an unpolarized incoming beam into the polarized cell):\n",
+    "\n",
+    "$$\\hat{T}^{unpol-in}(\\lambda, t) = T^{g, p|a}(\\lambda)e^{-O_0^{p|a}*\\lambda}\\cosh({P_{3He}(t)*O^{0, p|a}*\\lambda})$$<br>\n",


Avoid all the $*$? I think they don't help with readability.

docs/Data correction documentary-cleared.ipynb

Deletion of outdated file

docs/Data_correction_documentary_cleared.ipynb

jl-wynen · 2024-01-17T15:55:32Z

docs/Data_correction_documentary_cleared.ipynb

+    "\n",
+    "\n",
+    "Both $\\hat{T^p}$ and $\\hat{T^a}$ can be derived from their cell parameters (the transmission of the empty cell $T_g$,  and the cell opacity $O$), and the time-dependent nuclear polarization of the cell $P_{^3He}(t)$.<br>\n",
+    "This general method for time-dependent $^3$He-cells can be adapted for other use-cases such as using supermirrors or in-situ polarized ^3He-cells, see \"Note\" at the end of the page. \n",


This doesn't render properly: $^3$He

jl-wynen · 2024-01-17T16:00:12Z

docs/Data_correction_documentary_cleared.ipynb

+    "O^{cell}(\\lambda)= (0.0733 \\cdot p^{cell} \\cdot l^{cell} )\\lambda\n",
+    "$$\n",
+    "\n",
+    "with a factor 0.0733 arising from the number density of $^3$He gas and the absorption cross section of 1$\\mathring A$ neutrons by $^3$He. It can be seen that the opacity depends linearly on the neutron wavelength $\\lambda$. To calculate one value of $O(\\lambda)$ that fits to all wavelength from the depolarized direct-beam data $I^{DB, depol, cell}$, we will fit the wavelength-independent part $O^{0, cell}$ of $O(\\lambda)=O^{0, cell}\\cdot\\lambda$ in above equation. Afterwards, the wavelength-dependence can be retrieved by setting \n",


What do you mean by 'we will fit the wavelength-independent part'? Can't you just measure l and p and compute O^0?

EDIT: After reading on, it seems that everything up to here is the same for both workflows. So why is it in the 'Workflow 1' section?

we did not divide the workflows 1 and 2 into parts defined in the beginning of the script and end of the script, but into how they are physically derived.
In workflow 1, the opacity will be acquired by setting O=p l lambda = O_0 lambda, whereas in workflow 2 it will be acquired by the direct beam through the depolarized cell and fitting O_0 to the given function.
--> the workflows are the same everywhere except for how to derive O_0

for workflow 1 with setting O=p l lambda = O_0 lambda:
--> yes, O_0 is set by predefined p and l, that can be inserted by the sample environemtn team prior to the user measurement and will be just read-in values. still, O = O_0 lambda is wavelength dependent, and the lambda-dependency in how its written here is then treated the same as for workflow 2.
--> but for workflow 2, this does not work, O_0 has to be fit to the given function in workflow 2

So $O(\lambda)=O^{0, cell}\cdot\lambda$ is used in both workflows? Then why is it listed in the 'workflow 1' section? Can you move the common equations and text out of the workflow sections and cleanly separate the workflows into sections? This would make it easier to understand how things relate to each other.

Also, 'workflow 2' says

A more precise data correction is based on a measure of $p$ by a direct-beam measurement through the depolarized cell

Do you really measure $p$? It sounds more like you don't use the parametrisation with $p$ and $l$ in workflow 2 but treat $O^0$ as a fundamental parameter.

Thanks for this comment, it is a good idea to divide these information in a better way! I will upload a new version. I am also sorry for confusion, I meant $P(t)$ here, not p (pressure of cell). I have rewritten it.

jl-wynen · 2024-01-17T16:04:43Z

docs/Data_correction_documentary_cleared.ipynb

+    "\n",
+    "- Note: Workflow 2 should overwrite workflow 1 opacity after the experiment\n",
+    "- Note: Different to the sample data, $I^{DB,depol,cell}$ and $I^{DB,no-cell}$ are not time-dependent\n",
+    "- Note: It is important to fit $I_{depol}/I_0$ instead of $I_{depol}$, such that all the parameters in the fit-functions are scalars.\n"


What is I_0? I can't find a definition.

I forgot to change the name of this in the note, sorry. I_0 is now I^{DB, no-cell}_{\lambda}, I will change it

jl-wynen · 2024-01-17T16:05:26Z

docs/Data_correction_documentary_cleared.ipynb

+   "source": [
+    "#### Calculation of time-dependent $^3$He-polarization\n",
+    "  \n",
+    "Now $O^{cell}(\\lambda)=)O^{0, cell}\\cdot\\lambda$ is known. <br>\n",


Suggested change

"Now $O^{cell}(\\lambda)=)O^{0, cell}\\cdot\\lambda$ is known. <br>\n",

"Now $O^{cell}(\\lambda)=O^{0, cell}\\cdot\\lambda$ is known. <br>\n",

jl-wynen · 2024-01-17T16:08:01Z

docs/Data_correction_documentary_cleared.ipynb

+   "source": [
+    "#### Spin-dependent transmission values\n",
+    "\n",
+    "Now both $O^{cell}(\\lambda)=)O^{0, cell}\\cdot\\lambda$ and $P^{^3He, cell}(t)$ are known. <br>\n",


Suggested change

"Now both $O^{cell}(\\lambda)=)O^{0, cell}\\cdot\\lambda$ and $P^{^3He, cell}(t)$ are known. <br>\n",

"Now both $O^{cell}(\\lambda)=O^{0, cell}\\cdot\\lambda$ and $P^{^3He, cell}(t)$ are known. <br>\n",

SimonHeybrock · 2024-01-22T12:47:27Z

@astellhorn Can you fix the formatting or static analysis problems? This should actually happen locally when you commit (unless you bypass the check). You can run again using pre-commit run --all-files.

docs/Data_correction_documentary_cleared.ipynb

SimonHeybrock · 2024-01-22T12:58:15Z

docs/Data_correction_documentary_cleared.ipynb

+    "$\\mathbf{Workflow 2}$:<br>\n",
+    "A more precise data correction is based on a direct-beam measurement through the depolarized cell (i.e., $P_{^3He}(t) \\rightarrow 0$)(Ref. 2). For such a depolarized cell and an unpolarized incident beam (i.e., $\\hat{T}^{depol} = (\\hat{T}^++\\hat{T}^-)/2$), the neutron transmission becomes:\n",
+    "\n",
+    "$$\n",
+    "T^{depol, cell}(\\lambda)=T^{g, cell}(\\lambda)e^{-O^{0, cell}\\cdot\\lambda}\n",
+    "$$\n",
+    "\n",
+    "This leads to following direct beam intensity $I^{DB,depol,cell}_{\\lambda}$ measured on a depolarized cell:\n",
+    "\n",
+    "$$\n",
+    "\\frac{I^{DB,depol,cell}_{\\lambda}}{I^{DB,no-cell}_{\\lambda}}=T^{g, cell}(\\lambda)e^{-O^{0, cell}\\cdot\\lambda}\n",
+    "$$\n",
+    "\n",
+    "where $I^{DB,no-cell}_{\\lambda}$ is the intensity of a direct beam through an open neutron path (without cell and sample) and acts as a background measurement. For a measurement of $I^{DB,depol,cell}_{\\lambda}$, the <sup>3</sup>He cell needs to be depolarized at the end of its lifetime and $I^{DB,depol,cell}_{\\lambda}$ has to be acquired for each cell (and the other cell and sample out of beam). \n",
+    "\n",
+    "- Note: Workflow 2 should overwrite workflow 1 opacity after the experiment\n",
+    "- Note: Different to the sample data, $I^{DB,depol,cell}$ and $I^{DB,no-cell}$ are not time-dependent\n",
+    "- Note: It is important to fit $I_{depol}/I^{DB, no-cell}_{\\lambda}$ instead of $I_{depol}$, such that all the parameters in the fit-functions are scalars.\n"


The fact that we need to do a fit is hidden in a note. Can the procedure be explained more explicitly?

SimonHeybrock · 2024-01-22T13:27:26Z

docs/Data_correction_documentary_cleared.ipynb

+    "O^{0, cell}= 0.0733 \\cdot p^{cell} \\cdot l^{cell}\n",
+    "$$\n",
+    "\n",
+    "with a factor 0.0733 arising from the number density of <sup>3</sup>He gas, the path length $l$ of neutron through the cell, and the absorption cross section of 1$\\mathring A$ neutrons by <sup>3</sup>He. However, a measurement of the opacity by neutron transmission in a depolarized cell condition yields more precise values and can be performed at the end of the cell-lifetime during the user's neutron experiments, i.e., leading to \"workflow 2\". \n",


Is this 0.0733 missing a unit? I can't get the opacity math to work otherwise.

Yes, it has. To get this pre-factor according to the pressure [bar], cell length [cm] and wavelength [Angstrom] it is 0.0733[1/(kg/s2)] pro Angstrom. I mentioned it in the code of the branch calculating O, but yes of course I should better insert that here in the documentary: O is calculated from O=nsigmalambda*l ("n is the number density of 3He gas in the cell. σ0 is the absorption cross section of 1 Å neutron by 3He, σ0 = 2966 × 10−24 cm2. λ is the neutron wavelength. l is the path length through the cell. " - from Hals paper of the ECNS proceedings). Then exchanging with p = nk_BT. I thought its ok as its reported in other papers but its a quick copy-paste!

Yes, please update!

0.0733[1/(kg/s2)] pro Angstrom

Where is the Angstrom in that fraction, exactly?

If we can define things in terms of other constants that would be even better, avoiding magic numbers.

0.0733[1/(kg*Angstrom/s2)]

Please see my updates in #27.

Sorry about confusion. yes it is actually 0.0733 with units s2/kg per neutron wavelength - so 0.0733 (s2/kg*Angstrom) for 1Angstrom neutrons (which is typically done)

Yes I am having a look on the changes now - trying to keep track after coming back from USA! If I forget anything there please dont hesitate to remind me again.

SimonHeybrock · 2024-01-23T14:13:35Z

docs/Data_correction_documentary_cleared.ipynb

+    "\\begin{pmatrix} S^{++}_{t,\\lambda} \\\\ S^{+-}_{t,\\lambda}  \\\\ S^{-+}_{t,\\lambda} \\\\  S^{--}_{t,\\lambda}\\end{pmatrix}\n",
+    "$$\n",
+    "\n",
+    "- Note: probably easier to first put functions into the matrix product $T^a$ $\\cdot$ $T^p$, then numerically get the inverse of that matrix product.\n",


If done numerically this will be wasteful, since the matrices have a very special structure and many zeros. Therefore, it should be done analytically.

SimonHeybrock · 2024-01-23T14:15:46Z

docs/Data_correction_documentary_cleared.ipynb

+    "|  $O^{0, cell}$   |  wavelength-independent pre-factor of opacity of <sup>3</sup>He-polarizer/analyzer   |  scalar, 1-dim   | fit to data  |\n",
+    "|  $P^{^3He, cell}(t) = C^{cell}e^{-t/T^{1, cell}}$   |  time-dependent nuclear polarization of <sup>3</sup>He-polarizer/analyzer  |  continuous function of $t$   | $C^{cell}, T^{1, cell}$ fit to data   |\n",
+    "|  $C^{cell}$   |  Pre-factor of nuclear polarization exponential decay of <sup>3</sup>He-polarizer/analyzer  |  scalar, 1-dim   | fit to data | $P_{He0}$   |\n",
+    "|  $T^{1, cell}$   |  Decay constant of nuclear polarization exponential decay of <sup>3</sup>He-polarizer/analyzer  |  scalar, 1-dim   | fit to data |\n",


This appears to be a time-scale, can we use $t$ instead of $T$? As it is it is also easily confused with the other uses of $T$ for transmission.

Hm. Its a commonly used "T", as its not varying in time, its a fix constant. I feel it can be dangerous to exchange with 't' as well. I see the point of confusion with Transmission. Sometims these time-decays are also called "Gamma" (large greek character). I have no objection calling it "Gamma_1". But I believe I had called it "Gamma" before Hal said that for 3He cell neutron polarization its commonly "T_1". That may be a group decision here?

Hal recommends changing the Transmission from "T" to "Tr" instead, and leaving "T_1" as it is. Fine with me

… T_1 to Gamma_1

astellhorn · 2024-01-24T15:34:50Z

have let check run using pre-commit, but after it still complained about "Tung" to be changed to "tongue" (see Wai Tung Lees paper reference), I have used commit -n -m to commit it anyhow. How can I go around such an error message in a better way?

SimonHeybrock · 2024-01-25T05:58:23Z

docs/Data_correction_documentary_cleared.ipynb

+    "O(\\lambda)=n\\cdot \\sigma_0\\cdot l \\cdot\\lambda\n",
+    "$$\n",
+    "\n",
+    "with n being the number density of <sup>3</sup>He gas in the cell, $\\sigma_0$ the absorption cross section of 1 $\\mathring A$ neutrons by <sup>3</sup>He ($\\sigma_0 = 2966 \\cdot 10^{−24} cm^2$), and the path length through the cell $l$ [Ref. 1]. As <sup>3</sup>He can be considered to follow the ideal gas law, $n$ can be replaced via $p=n\\cdot k_B\\cdot T$:\n",


Do we have an error bar for the 2966? Then we could include it in the propagation of uncertainties.

SimonHeybrock · 2024-01-25T06:21:48Z

have let check run using pre-commit, but after it still complained about "Tung" to be changed to "tongue" (see Wai Tung Lees paper reference), I have used commit -n -m to commit it anyhow. How can I go around such an error message in a better way?

I added it to the spell checker's ignore list for you. Run git pull to update your local branch.

Data correction documentary

4ee5990

Documentary for functions and dependencies

astellhorn assigned SimonHeybrock and nvaytet and unassigned SimonHeybrock and nvaytet Jan 12, 2024

astellhorn requested review from SimonHeybrock and nvaytet January 12, 2024 12:41

SimonHeybrock reviewed Jan 12, 2024

View reviewed changes

astellhorn added 2 commits January 15, 2024 15:50

Updated documentary

71f4ead

Delete docs/Data correction documentary-cleared.ipynb

131ffd2

Deletion of outdated file

astellhorn mentioned this pull request Jan 16, 2024

Document maths and physics of polarization analysis #16

Closed

SimonHeybrock requested a review from jl-wynen January 17, 2024 14:20

jl-wynen reviewed Jan 17, 2024

View reviewed changes

astellhorn added 2 commits January 17, 2024 22:37

Update Data_correction_documentary_cleared.ipynb

c0fe376

further changes

ee4c0b3

jl-wynen approved these changes Jan 22, 2024

View reviewed changes

SimonHeybrock reviewed Jan 22, 2024

View reviewed changes

SimonHeybrock mentioned this pull request Jan 22, 2024

Implement and test updated opacity definition #27

Merged

SimonHeybrock reviewed Jan 23, 2024

View reviewed changes

changed opacity description, but still waiting for approval to change…

bc5901c

… T_1 to Gamma_1

SimonHeybrock reviewed Jan 25, 2024

View reviewed changes

Add name to spell checker ignore list

55dd560

SimonHeybrock mentioned this pull request Jan 31, 2024

Error bars for the He3 cross section? #28

Open

SimonHeybrock merged commit 648383b into main Jan 31, 2024

SimonHeybrock deleted the astellhorn-patch-2 branch January 31, 2024 09:46

	"Now $O^{cell}(\\lambda)=)O^{0, cell}\\cdot\\lambda$ is known. <br>\n",
	"Now $O^{cell}(\\lambda)=O^{0, cell}\\cdot\\lambda$ is known. <br>\n",

	"Now both $O^{cell}(\\lambda)=)O^{0, cell}\\cdot\\lambda$ and $P^{^3He, cell}(t)$ are known. <br>\n",
	"Now both $O^{cell}(\\lambda)=O^{0, cell}\\cdot\\lambda$ and $P^{^3He, cell}(t)$ are known. <br>\n",

Data correction documentary #26

Data correction documentary #26

Uh oh!

Conversation

astellhorn commented Jan 12, 2024

Uh oh!

SimonHeybrock left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

astellhorn Jan 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SimonHeybrock commented Jan 22, 2024

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

astellhorn Jan 12, 2024 •

edited

Loading