Author: Henry Rzepa

Back in early 2012, I pondered about the relationships between a science-based blog post and a science-based journal article[cite]10.59350/3pbz1-vcd67[/cite]. This was in part induced by my discovering a blog plugin called Kcite, which allow a journal articles to be appended to the blog in the form of a numbered reference list. The only required input for Kcite was the DOI of the article (as you can see earlier in this paragraph). For around 500 posts after that moment, I always strove to add such references to my posts. Around 2016, I started including references to data in the form of repository DOIs to sit alongside the journal references, but this feature stopped working a year or two later because of changes in the metadata resolved by the DOI. Kcite itself lasted until January 2024 for this blog, when a required update to the software running the blog (WordPress) meant that it no longer worked and had to be removed as a plugin. Two years ago, Rogue Scholar (Science blogging on steroids) started coming along to the rescue.[cite]10.53731/axtz227-73n18e7[/cite] ,[cite]10.53731/4bvt3-hmd07[/cite] It provides some amazing automated features and infrastructure to blogs; I will illustrate from those listed on the top page of Rogue Scholar itself:

1. No waiting time — blogs can join via a simple form. Blog posts are automatically archived within minutes after publication on your blog.
2. No fees — blog posts are archived without fees to either readers or authors. Rogue Scholar is sustained by donations and sponsorships.
3. Archived — blog posts are archived by Rogue Scholar, and semiannually by the Internet Archive Archive-It service.
4. Findable — every blog post is searchable via rich metadata and full-text search.
5. Citeable — every blog post is assigned a Digital Object Identifier (DOI), to make them citable and trackable. Rogue Scholar shows citations to blog posts found by Crossref.
6. Interoperable — metadata are distributed via Crossref and ORCID, and downstream services using their metadata catalogs.
7. Reusable — the full-text of every blog post is distributed under the terms of the Creative Commons Attribution 4.0 license.
8. Communities — blog posts automatically become part of communities for your blog, the blog subject area, and topic communities based on blog post tags.

Part of the stuff that goes on behind the scenes is integration with CrossRef (which handles information about journal articles) and that in turn enables insights such as how Blogs abstracted by Rogue Scholar can be cited within journal articles and other blogs and gives some idea of the impact that these blogs are making. Here I illustrate some searches so enabled by having Rogue Scholar abstract a blog;

1. https://rogue-scholar.org/search?q=references:*&sort=newest This shows that Rogue Scholar has captured (currently) 2003 references abstracted from blogs.
2. https://rogue-scholar.org/communities/rzepa/records?q=references:*&sort=newest Of these (currently) 504 have come from mostly identifying the [cite]…[/cite] entries in my own blogs.
3. https://rogue-scholar.org/search?q=citations:*&sort=newest shows all citations of the blogs in the Rogue Scholar community, currently at 519.
4. https://rogue-scholar.org/search?q=citations:10.59350/*&sort=newest This lists the number of citations originating from the  DOI prefix 10.59350 (which is that of the Rogue Scholar community).
5. https://docs.rogue-scholar.org/dashboard lists other statistics. This are revealing, indicating currently only 6% of posts currently have references, although the uptake of institutional origins (ROR) and researcher ID (ORCID) is much better.
   
6. The distribution amongst subject areas is 6.8% in the chemical sciences:

Meanwhile, work is under way to resuscitate the Kcite plugin, so that references are once again collected at the bottom of each post. Meanwhile, such a list can instead be found at the archived version of the posts at Rogue Scholar, as for example for this post itself. Also for the future is identifying how many of the references cited in blogs relate to research objects such as journal articles, and how many are instead to data held in e.g. data repositories. Such data reference richness in journal articles themselves is gradually increasing[cite]10.59350/th26w-gev67[/cite],[cite]10.1039/D3DD00246B[/cite] and it to be hoped also in science-based blogs themselves in the future.

X-ray crystallography is the technique of using the diffraction of x-rays by the electrons in a molecule to determine the positions of all the atoms in that molecule. Quantum theory teaches us that the electrons are to be found in shells around the atomic nuclei. There are two broad types, the outermost shell (also called the valence shell) and all the inner or core shells. The density of the core electrons is much higher (more compact) than the more diffuse valence shell for all but the hydrogen atom, which only has valence electrons. How does this relate to x-ray diffraction by electrons? Well, core electrons, because of their relative compactness, diffract X-rays more strongly than the valence electrons. This compactness of the core also means that its electron density distribution can be well (but not exactly) approximated by a sphere, with the nucleus at the centre of that sphere. And from this it follows that the density for each atom can be treated independently, the so-called IAM or independent atom model. For example all the carbon atoms in a molecule are approximated as having the same value for the electron density of their core shell. But the IAM approximation is much less good for hydrogen atoms, especially when they are attached to very polar atoms (Li, O, F, etc) and even atoms such as carbon or oxygen have noticeable deviations as illustrated in  figure 1 below. [cite]10.1039/d0sc05526c[/cite]

Figure 1 from [cite]10.1039/d0sc05526c[/cite] with caption: Deformation Hirshfeld densities for the carbon (left) and oxygen (right) atoms in the carboxylate group of Gly-l-Ala, i.e. difference between the spherical atomic electron density used in the IAM and the non-spherical Hirshfeld atom density used in Hirshfeld atom refinement=HAR (IAM minus HAR). Red = negative, blue = positive. Isovalue = 0.17 eÅ⁻³.^‡

X-ray crystallography is all about matching the electron density map of a model structure with the electron density map derived from the diffraction data. In “conventional” X-ray crystallography – i.e. that used by most crystallographers –  the electron density map of the model is calculated using the IAM approach, where no consideration is given to any distortion of the electron density distribution caused by things like bonds – each atom is treated independently (hence the name). This method especially struggles with hydrogens and hence the inferred position of the hydrogen nucleus at the centre of an assumed spherical distribution is often difficult to obtain accurately. Enter quantum crystallography, whereby a model of the electron density distribution in a molecule can be calculated by solving the Schrodinger equation, nowadays to a very reasonable approximation in a reasonable time (minutes) using so-called density functional theory, or DFT. The resulting electron density map for the model structure might be expected to more closely match reality than the IAM approach. Most obviously affected by this change is the handling of hydrogen atoms. If one considers a C–H bond from an sp³ carbon atom, using an IAM approach the hydrogen atom (i.e. its nucleus or proton) would be placed at the centre of maximum electron density, in the full knowledge that this is not actually where the hydrogen atom nucleus itself is. The direction of the C–H vector would be correct, but the distance would be too short. In the quantum crystallography approach, the positions of e.g. hydrogen atom nuclei are not exactly coincident with the electron density maxima, amounting in effect to non-spherical atoms, thus avoiding the systematic errors seen in the IAM approach. Smaller, but possibly still significant such errors might be expected for e.g. the 2nd row elements and beyond.

Getting reliable hydrogen atom positions has previously required a neutron diffraction study, which is difficult, expensive and time consuming. So the idea of using the non-spherical DFT densities rather than the spherical IAM approach to build a model using X-ray diffraction data is very appealing. But does it work? To test this, we decided to go back to some previously published structures that were handled using the IAM approach, and re-refining them using quantum crystallography. We do not have the corresponding neutron studies to check the answers against, but we can still see how well the structures themselves refine and what new problems this approach might throw up.

Method

The original published structures[cite]10.14469/hpc/2297[/cite] were refined with SHELX-2014[cite]10.1107/S2053229614024218[/cite] which uses an independent atom model (IAM) approach. The results reported here employed NoSpherA2[cite]10.1039/d0sc05526c[/cite], [cite]10.1107/S0021889808042726[/cite] using Hirshfeld atom refinement[cite]10.1107/S2052252514014845[/cite] and selecting Def2-SVP as the (all-atom) basis set^† and ωB97X-V as the DFT method (the results seem relatively insensitive to either), implemented in the ORCA program.[cite]10.1002/wcms.81[/cite] For the first attempts no changes were made to the structures beyond the anisotropic refinement of the now unconstrained hydrogen atoms. For four of the structures a number of the hydrogen atoms went non-positive definite (i.e. one of the radii of a thermal ellipsoid refined to a negative length), which is physically nonsensical and would be a significant barrier to publication. (we don’t quite want to say “unpublishable” as there are almost always exceptions, but a non-positive definite thermal parameter is pretty close to being unacceptable.) For these cases, a second version was created (V2) where all of the hydrogen atoms were refined isotropically but with the distances and thermal parameters still allowed to refine. For AB1709 (18b), this still had the isotropic thermal parameter of one of the hydrogen atoms (H11) go non-positive definite, so for that one hydrogen atom the free isotropic thermal parameter was replaced with a riding one.

The results

We chose a set of seven structures published in 2017[cite]10.1021/acsomega.7b00482[/cite] and refined as noted above using conventional methods. These seven also comprise one of the very first sets of crystal structures for which full diffraction data were made available,[cite]10.14469/hpc/2297[/cite] rather than just the refined structure in the form of a CIF file. The  new results have also been deposited[cite]10.14469/hpc/15030[/cite] to augment the record for these compounds. Spreadsheets corresponding to the images below can be obtained by clicking on the image.

1. All seven structures saw a reduction in the final R-factor.[cite]10.14469/hpc/15030[/cite] However, all of the structures also saw a significant increase in the number of parameters (as the hydrogen atoms went from using zero parameters each in a fully riding model to nine parameters each in a fully free anisotropic model). However, all the QM refinements passed the Hamilton test, suggesting that the reduced R-factors do indeed reflect a better model, rather than just being a consequence of the significantly increased number of parameters.
2. All four of the structures containing bromine atoms had a number of the hydrogen atoms go non-positive definite when refined anisotropically. It is not clear exactly why this happened – there does not appear to be any correlation with data quality or intensity (as crudely measured by R(int) and mean I/σ respectively), and though the redundancy for these structures is fairly low (between 1.5 and 1.7), those for the non-bromine structures are not much better (1.5, 2.3 and 4.9). These data sets were the result of experiments designed to collect 98.5% of the symmetry unique data with no consideration for redundancy at all. However comparison of the initial and secondary versions of the refinements of these four structure does show that the substantial majority of the observed R-factor decrease can be achieved without using anisotropic hydrogen atoms.
3. As regards the precision of the structures, using one C(sp2)–C(sp3) bond as a proxy (the C7–C8 bond) we can see that the estimated standard deviation is either the same or only slightly lower in all seven structures, suggesting that getting lower e.s.d.s would not be a motivating factor for using quantum crystallography.
4. One of the more unexpected results was the variation in F(000). In X-ray crystallography (deliberate emphasis on X-ray, as neutron diffraction is different) F(000) is supposed to be the total number of electrons present in the unit cell, and is used as an overall scale factor for the electron density map. It is very much not supposed to be variable, and any discrepancy would indicate an error in the calculated or reported formula and should be corrected. We do not understand why the QM refinements give a different answer than the IAM ones (some up and some down — normalised to a per molecule basis the range is –1.1 to +2.2), though it seems likely to be associated with cut-offs (boundaries) in measuring the “smeared out” electron density in the QM models, The IAM models all give the expected “correct” values.
5. Based on the checkCIF reports for the QM structures, if quantum crystallography catches on in a big way, then checkCIF will probably need to be updated, there now being a number of high level alerts for long X–H bonds.
6. One of the major areas of uncertainty with quantum crystallography is what/how much data needs to be collected. Symmetry unique data to 0.84 Å seems insufficient, but what would be sufficient — full sphere, redundancy, higher resolution? Would the final results be worth the extra time investment? None of the above aspects are clear at this stage, but it will be interesting to see how the technique develops.

These seven crystal structures also occupy an interesting position for posterity. Data for them has been made available spanning eight years which illustrates two significantly different refinement methods being used during this period, as well as having access to  the original complete diffraction image data to allow any completely new analysis to be made in the future.  Who knows, maybe in eight years time an even better method may become available for comparison with the results reported here.

^‡To put this into context, 0.17 eA^-3 would generally be regarded as a pretty low level background noise, similar to the value of the maximum residual electron density a crystallographer might be happy with. ^†The structure which showed the smallest change in R factor on using quantum crystallography, i.e. AB1608b, was re-run with the triple-ζ Def2-TZVPP basis set. This did give lower R factors but by very little (3.38% to 3.36% aniso with npd; 3.39 to 3.38 iso).

Starting around 2016, journal publishers started including mandatory “Data Availability” statements as part of research articles; a typical (dated) example is linked here, including guidelines for how to cite the data itself. I wrote about these aspects last year in a blog post for the RSC journal Digital Discovery[cite]10.26434/chemrxiv-2024-dz2dv[/cite] and here I follow up with more news.

In a recently published article about Direct Amidation Reactions[cite]10.1039/D4SC07744J[/cite], the following version of a data availability statement appears: An IUPAC FAIRSpec Finding Aid for the NMR spectroscopic data is available at DOI: 10.14469/hpc/14884. A selection of data discovery searches can be found at DOI: 10.14469/hpc/14822 and it introduces the concept of a Finding Aid. Put simply, knowing where the data supporting a research is available will not necessarily lead you to the particular datum you might be looking for, especially if there is a lot of data. Data is still frequently made available in the form of a supporting document called ESI, and such documents can contain many tens of compounds and possibly hundreds of associated spectra. The aim of a Finding Aid is to help you find the ones you are interested in.

If you are interested in how this works, go explore either of the two links given above.  The Finding Aid tool was created by Bob Hanson as part of an IUPAC working party on how to create spectroscopic data in so-called FAIR form (The F of FAIR and the F of Finding Aid are one and the same of course!). This represents its first deployment for a newly published article. The creation tool itself is still α-stage – further tools are being developed – of which more later.

The idea of so-called FAIR (Findable, Accessible, Interoperable and Reusable) data is that each object has an associated metadata record which serves to enable the four aspects of FAIR. Each such record is itself identified by a persistent identifier known as a DOI. The trick in producing useful FAIR data is defining what might be termed the “granularity” of data objects that generate the most readily findable and which most usefully enable the other three attributes of FAIR.

To set the scene for how to do this optimally, I first set out two extreme examples of FAIR objects relating to chemical spectroscopy such as NMR. These will be directly associated with a journal article describing for arguments sake say 50 compounds new to science, with the existence of these data objects identified via a data availability statement appended to the article. Each compound might be characterised by say spectroscopic and crystallographic information and perhaps some computational analysis. For the spectroscopic analysis, perhaps 5 types of NMR experiments might be included, giving a total of around 10 separate types of datasets for each compound, or in round numbers lets say 500 data sets for the 50 compounds reported in such an article.^†

• Method A: The data associated with an articles takes the form of a ZIP (or other type of compressed) archive containing all 500 of the intended FAIR data sets. The resulting ZIP file is then described with a single metadata record and assigned a single DOI using e.g. the tools of a data repository. That one metadata record has the (mammoth) task of describing all of these datasets, across perhaps ten different kinds of experiment. This type of monolithic object is in fact not unusual, for several reasons. Some repositories impose a significant charge for each deposition, and so the temptation to reduce costs would be to adopt this expedient.
• Method B: The other extreme is to literally deposit all 500 data sets separately and assign 500 DOIs, each with a separate metadata record. The issue now is less how well the metadata record can describe each dataset, but more of to establish the relationships between these 501 objects (the journal article and each dataset). Such relationships could include:
  • that between the compound molecular structure and the dataset
  • that between say the dataset and the type of spectroscopic experiment (e.g. IR, MS, NMR, XRD, Comp)
  • that between different eg NMR experiments for the same compound (the nucleus, the pulse sequence, the solvent, etc).
  • These could in total represent a great many individual relationships between both the 500 data sets and the article itself (formally around 501²/2!)

Before setting our solution, I show below how a typical repository such as Zenodo handles the relationships between data objects noted above.

The relation type is selected from a controlled list of about 30, and is entered for each individual metadata record associated with a DOI. So clearly, relationships in the second category would have to be individually entered, hardly feasible for 501²/2 entries. And in the first category, only one relationship between the single large archive of data and the journal DOI can be added. One of the  more important relationships in this context are the “Has part”  or “Is part of” ones (diagram above).

The use of this now constitutes Method C.

1. One starts by creating what could be called a top or level 1 entry, which will contain important  core metadata information such as the contributing authors, the institute where the data was obtained,  the title and overall description of the datasets to come, a license,  a date, a declaration of the published article associated with the data and finally the  DOI of this metadata record. This top-level entry would  also list all the compounds on level 2 for which data is available  and each being referenced by a “Has part” declaration via a  DOI for each compound.
2. Each compound on level 2 would in turn point back to level 1 by an “Is part of” metadata declaration. Each compound on level 2 would also  list the spectroscopic experiments available that compound, for example the NMR method as part of level 3. It would have an “Is part of” declaration pointing back to the compound level  2 entry.
3. The  list of the different NMR experiments on  level 3 also have “Has part” declarations pointing  to the list of NMR experiments on level 4.
4. Each NMR experiment conducted on level 4 would contain an “Is part of” declaration back to level 3 and a list of “Has part” entries which describe the individual data files available for that experiment in the metadata record for level 4.

If you wish, you can inspect all “Has part”/”Is part of” declarations in the metadata records for these various levels by invoking e.g. https://data.datacite.org/application/vnd.datacite.datacite+xml/10.14469/hpc/11446 (replacing e.g. 11446 by any of the DOI suffixes shown in red in the diagram below). They are all associated with this published article.[cite]10.1021/acs.inorgchem.3c01506[/cite]

What does this use of relational parts declarations achieve? Well, compared to method  A, where everything had to be achieved within a single metadata record (and in practice never is) or method  B, where a very large number of relationships would have to be declared (and again never are), Method C achieves a good balance between the two.^‡ By collecting the metadata information into groups, one can achieve a more readily navigable structure for the information and also allow sub-groups to effectively inherit properties from the higher group.

I end by noting that far too few FAIR data collections associated with published journal articles adopt such procedures, in large part because of very little current exploitation of relationships between the data such as the one used above (“Has part”/”Is part of”). The repository itself has to  be carefully designed to do this as automatically as possible and not require the human depositor to invoke each instance by hand (as shown for e.g. Zenodo above). An example of just such a repository is described here.[cite]10.1186/s13321-017-0190-6[/cite]

^†The data sets themselves might be made available in more than one form (for NMR, a Bruker ZIP archive, an Mnova file, a JCAMP-DX format or just a PDF spectrum), thus increasing the number even further.
^‡It reminds me of when I used to teach molecular orbital theory using the Hückel method, which requires a secular matrix to be diagonalised. For e.g. naphthalene, this operation would have to be conducted on a 10*10 matrix, something almost impossible by hand. However, one could use group theory to block diagonalise this matrix into much smaller matrices with the off-diagonal elements between them set to zero, thus considerably reducing the task at hand.