Interactions with the Therapeutic Goods Administration

How I managed to get something via a Freedom of Information request

This article follows from the previous one about vaccine batches and reporting of adverse events.

I had been informed by the Therapeutic Goods Administration (TGA) that it was not possible to get the information I was requesting in FOI-3545 (vaccine batch/lot numbers associated with Adverse Events (AEs)) without significant modifications to the Adverse Event Management System (AEMS) user interface that the TGA uses. It turns out that there is a custom user interface sitting between the AEMS database and the user. The database is apparently not SQL compatible (I was hoping that if I wrote down the query for them, then they might be able to just change a couple of terms and give me what I needed). Apparently, it is not possible to directly query the database, all information has to go through this custom user interface. Two people at the TGA informed me that it was not possible to get the batch information from AEMS and have it linked to the case number in DAEN. I suspect that was true at the time, but possibly my persistence, and my request ending up in the hands of the right person resulted in a favourable outcome.

My first FOI request with the TGA was poorly done. I asked for too much information. It was rejected for multiple reasons including asking for the date of death, as the information could be linked to an individual (age, sex and date of death). When I attempted again, I asked for a lot less and provided backup options to cover for the possibility that what I was asking was impossible. It was initially rejected, but I asked a few more questions and then understood the issue more (their user interface limited what you could access). I was then under the impression that the best I could get was just a list of batch numbers that were associated with a suspected AE from a COVID-19 vaccine (not linked to a case number). Not ideal, but better than nothing. Because my request included the following text, I didn't need to submit a new request, just point out that it was possible to provide something useful.

Alternatively, if the information is not stored in this fashion, then please provide whatever batch code information is available.

When a request is rejected, you can ask for an internal review. I did that, and pointed out that what I was requesting was possible. At the same time, I submitted a new request for batch numbers for just serious AEs and just deaths – it turns out that wasn't possible either, their system seems extremely limited and inflexible.

The internal review is performed by someone at a higher level of management and I was extremely lucky to get someone who probably looked at my request and thought "Why can't we get that information? We should be able to do that for our own analysis" (wild speculation on my part). I was shocked when I received the spreadsheet and it had the case number (an ID that matches DAEN reports). This apparently was not possible and yet there it was.

I performed some analysis, with particular interest in Important Medical Events (IMEs) to get rid of some of the less important AEs like a headache or sore arm. As soon as I was confident with my initial analysis I contacted the TGA asking for them to pass information to a statistician at the TGA. My message to the TGA is below (with personally identifiable information deleted):

Dear TGA info,

Could you please pass this email on to a statistician at the TGA.

I have acquired information about batch numbers and adverse event reports from the TGA via FOI 3545. I have analysed this data set, matching records to reports in DAEN (Database of Adverse Event Notifications). I have seen significant differences in the number of Important Medical Events (IMEs as defined by the European Medicines Agency) across different batches. The reason for using IMEs is to exclude variability from relatively unimportant Adverse Events (AEs) such as a headache or sore arm. I have not yet been able to obtain the number of doses administered per batch. This information was collected for me in the Department of Health FOI 3467, but the FOI request was ultimately unsuccessful. The document created by the Department of Health is called 'COVID-19 Vaccine Batch Data Extract 12/01/2022'.

I'm not sure if I will ever be able to access the doses administered per batch information. I would appreciate it if a statistician could have a look at this and determine the magnitude of the problem or if there is any problem at all.

A significant difference in the number of IMEs per dose in one batch relative to another could indicate the following issues:

1. A difference in the safety of different batches

2. A difference in the reporting rate of IMEs across different batches

3. A difference in the reporting rate of the batch codes across different batches (might be reported in AEMS, but without batch code information)

4. A difference in duplicate or fraudulent entries across different batches

5. Some combination of the above

All of these indicate a significant problem. If the problem is due to safety issues in different batches, then vaccination should be halted immediately until the cause is determined. If the problem is related to reporting rates, then the highest reporting rate per batch could be used as a benchmark to scale up the reporting rates on all other batches. This might provide a more accurate estimate of the rate of occurrence of serious problems such as myocarditis.

I performed the analysis by including any dose in the series that was recorded. Less than 2% of records had more than one dose entry for a TGA case number.

I have attached the following files:

TGA Adverse Event Case, Batch and Dose In Series data.xlsx

The original spreadsheet provided by the TGA through FOI 3545

FOI_3545_batch_numbers_edited.txt

The file I used for analysis with modifications to some batch numbers that did not match known batch numbers.

Pfizer batches with AEs and IMEs.xlsx

This file has a list of batches where comirnaty was suspected, with the number of IMEs and all AEs recorded in AEMS/DAEN. It also includes an age summary of reports with IMEs to see if there was any link between IMEs and age (but without doses per batch information this is not possible). Two graphs are included. One comparing the number of IMEs and AEs recorded per batch, some batches stand out from the seemingly linear relationship between (0,0) and FF4206, with ER7449 being the significant outlier (along with EP9605 and EP2163). Perhaps these batches had much better reporting rates as they were the first and were subject to greater scrutiny, or perhaps there was a higher rate of medical professionals in these batches who know how to report issues.

I am fairly confident that the Pfizer batch code entry errors are minimal in my txt file, I am not confident of the errors in batch codes from other manufacturers. I used a Levenshtein distance of two to automate the process to match with a list of known batch codes, and I also manually edited some where the error seemed clear.

I would be happy to discuss this via email [deleted] or phone [deleted]. I would really appreciate it if I could know that a statistician is taking the time to investigate this and using the number of doses per batch information to determine the relative safety (or relative reporting rates) of different batches. I am under the impression that the TGA was not internally able to access this information (batch codes linked to AEMS reports) until now.

Regards,

[deleted]

The response I got seemed to be a standard reply for any questions or comments about vaccine batches.

Dear [deleted],

Thank you for your email to the Therapeutic Goods Administration (TGA) on 6 April 2022.

Ongoing assessment of vaccine safety is an integral part of the provisional registration process and the TGA is closely monitoring adverse events in Australia. The TGA has robust procedures in place to detect and investigate signals for possible safety concerns with COVID-19 vaccines. If a safety concern is detected, rapid action will be taken to address the safety issue and promptly provide information to the public including in the TGA’s TGA weekly COVID-19 vaccine safety report.

The TGA also undertakes an independent quality assessment of every batch of vaccine supplied in Australia to ensure it meets strict quality standard. Post-market surveillance data on global real-world use of COVID-19 vaccines continue to provide reassurance about their longer-term safety. This is supported by reviews of safety data by international medicines regulators in countries with extensive COVID-19 vaccine experience.

The TGA also ensures there is an independent quality assessment of every batch of vaccine supplied in Australia through vaccine batch release assessment. For all COVID vaccines supplied through the government’s vaccination program, a batch release assessment includes a review of documents supplied by the sponsor which describes the manufacturing process; TGA laboratory testing (and/or review of testing results from an overseas regulatory laboratory that has been recognised by the TGA). The results of this assessment are publicly available here: https://www.tga.gov.au/batch-release-assessment-covid-19-vaccines. This link provides guidance on the batch assessment process as well as a table of information showing the test results for each batch of COVID-19 vaccine that has been tested by the TGA.

Tests are performed on a variety of the vaccine’s properties, including the ingredients in the product (composition, strength, and identity) and the absence of impurities (purity, integrity, and endotoxin). The testing results, along with the review of the manufacturing documentation for each batch, provides assurance that the vaccine being supplied is in line with registered products on the Australian Register of Therapeutic Goods (ARTG).

We thank you for your interest in the work of the TGA and would like to reassure you of our commitment to the safety, quality and efficacy of all medicines approved for use in Australia.

Thank you for your understanding.

Yours sincerely,

[deleted]

TGA Contact Centre

They couldn't even be bothered getting a statistician to write a reply indicating they looked at what I sent. Unfortunately, I will probably never get the doses administered per batch information, so I won’t know the magnitude of differences in reported AEs per batch.

Here are the results of the analysis of the Pfizer COVID-19 vaccine batches. AEs refer to the number of cases per batch with any Adverse Event. IMEs refers to the number of cases which contained an Important Medical Event. The age summary statistics information was a crude check that the age distribution was not a likely cause of differences. I left out batches that had really small numbers of AEs because they started to include batch numbers that were not for Pfizer batches.

Here is the file I used for analysis, with obvious errors in batch codes repaired. I focussed on Pfizer batch codes. There may be a lot of errors in batch codes from other manufacturers as I didn't spend a lot of time on that.

Some other comments that might be useful to someone making a FOI request.

• A Department of Health or TGA FOI request takes at least 30 days. I get the impression that if you asked for the simplest thing, then they would collect it and then wait to release it at the deadline.

• If something you ask for requires consultation with 3rd parties then that adds an extra 30 days.

• It is worth putting in backup options in the request.

• Multiple small requests are better than one big request (better chance of getting something you want rather than nothing at all).

• The Database of Adverse Event Notifications (DAEN) is a subset of AEMS. The TGA internally uses AEMS, and DAEN is the publicly accessible portion.