The community of independent researchers is divided about where a new underwater search should be conducted. In this light, Bobby Ulich, PhD, has been quietly developing new statistical criteria that supplement the criteria that many of us have used in the past. The objective of this research is to define a new search area that is of manageable size and has a reasonable chance of success. Bobby has been working closely with Richard Godfrey to identify candidate paths, and with me to a lesser extent. At this point, the most promising candidate is a path of constant longitude, i.e., following a constant track of due south, aligned with waypoint BEDAX, and crossing the 7th arc near 34.3S latitude. This path, which was identified as a candidate path some years ago, could have been easily programmed into the flight computers with the waypoints BEDAX and the South Pole (e.g., by entering the waypoint S90E000). Work continues to evaluate this and other paths.
The note below from Bobby gives an overview of the new methodology, and is provided here to elicit comments from contributors of the blog.
Note from Bobby Ulich, PhD
There are two pieces of available MH370 information that have not yet been fully utilized in predicting the southern route and the most likely impact latitude near the 7th Arc. Neither one was included in the DSTG Bayesian analysis.
The first item is the inclusion of a detailed and accurate fuel flow model. All DSTG analyses effectively ignored fuel consumption as a route discriminator by assuming “infinite fuel”. I and several others subsequently developed fuel flow models and compared them with Boeing tables, prior 9M-MRO flights, and the MH370 flight plan in order to validate the predictions. My model fuel flows are consistent with these comparisons, nearly always within +/- 1 %. As I understand it, Boeing did some range and endurance calculations and found that it was possible, at some combination of altitude and speed, for 9M-MRO to have reached a wide stretch of the 7th Arc, totally excluding only the region beyond circa 39S. However, the DSTG analyses did not factor in fuel consumption for each route they examined, so it is unknown whether or not that route was actually flyable with the available fuel. This is an important consideration, especially considering the fact that over a large portion of the SIO region of interest, the high-altitude air temperatures on the night in question varied by as much as 12 C with respect to ISA over an altitude range of only about 5,000 feet. Thus, small altitude changes had major effects on TAS and on fuel flow.
In my route fitter, one can either predict the main engines fuel exhaustion (MEFE) time using the known average cruise PDA for the engines, or hold the MEFE time at 00:17:30 (per ATSB) by adjusting the assumed PDA. I generally do the latter because that gives the correct weight as a function of time, which affects the commanded airspeed. It is also important to allow for the possibility that bleed air was turned off after diversion, reducing fuel flow by several percent. So, in fact the effective PDA which would give the correct endurance is potentially a range of values between 1.5% (the known value with bleed air on) and about -0.8% (the equivalent of bleed air off for the entire flight after diversion). Best-fit PDAs higher than +1.5% and lower than -0.8% are increasingly less likely.
Inmarsat and DSTG have provided some analyses of the BTO and BFO reading errors based on prior flights. Generally speaking, up to now, the statistics associated with those reading errors are the mean and the standard deviation. In many cases the RMS statistic has also been used as a convenience when performing route model fits to the satellite data, but the fundamental statistics are the mean and standard deviation of the BTO/BFO reading errors. In the case of the BFOs, we cannot know that the mean reading error (effectively the BFO bias term) did not change as a result of the in-flight power cycle ending circa 18:24. Therefore, we really only have three of these four statistics available for route fitting – the BTO residual mean, the BTO residual standard deviation, and the BFO residual standard deviation.
Regarding the standard deviation of the BFO reading errors, there has been much discussion in the past regarding the seemingly inconsistent criteria used by Inmarsat and by DSTG. I have been modeling the BFO reading errors comprising (1) random electronic noise, (2) non-ergodic and non-stationary OCXO drift, (3) trigonometric quantization errors in the AES Doppler compensation code that give the appearance of being quasi-random for readings widely spaced in time, and (4) vertical speed errors of roughly 40 fpm or higher from the nominal vertical speed profile (caused primarily by turbulence). I have developed a simple statistical model for the BFO reading noise, and this model gives results which are consistent with both sets of criteria. This new BFO reading noise model is what I use in my current route fitter.
The second item of previously unused information is also significant in discriminating against incorrect routes. That item is the fact that the BTO and BFO reading errors are uncorrelated with themselves and with each other, and, in effect, with most route-fitting parameters (with possibly one exception). Note that it is the reading errors which are uncorrelated, not the actual expected values for BTO/BFO. The degree of correlation of pairs of parameters is quantified using the Pearson correlation coefficient, and these values may be used as additional statistical metrics of the goodness of fit for the True Route. The one exception that could have occurred is for the BFO reading errors, which could change linearly with time due to OCXO drift. So, we must use that one correlation case (BFO residuals with respect to time) cautiously, knowing that it is possible such a drift might have occurred, although we do not see obvious cases of large linear drifts in previous flights. It seems possible that a very cold cycling of the OCXO might produce a shift in the bias frequency, but it is not apparent that such a cycling would be more likely to produce greater linear drift of the bias frequency well after warm-up. Small drifts of several Hz during a single flight are seen on prior flights, and it would not be surprising if they occurred in MH370 after 19:41.
We can substantially increase the number of statistical metrics that must be satisfied by a fit using the True Route by adding numerous correlation coefficients to the previously used means and standard deviations of the BTO and BFO residuals. Using more metrics (~10-12 total) provides greater selectivity in route fitting and better discrimination against incorrect routes.
A requirement for effectively utilizing these additional correlation metrics is the development of a method which allows us to obtain best-fit BTO/BFO residuals which behave statistically the same as the BTO/BFO reading errors. Those two parameters, the best-fit residuals and the reading errors, are generally not the same quantity. They are the same only when the model used to predict the aircraft location in 4-D (with 7 assumed route parameters) is perfect. No 4-D aircraft prediction model is perfect, especially in light of the expected errors in the GDAS temperature and wind data (which must be interpolated in 4-D for calculations along the route). At best, there will be small (a few NM) systematic errors in the model predicted position at the handshake times. The along-track and cross-track components of the position errors are systematic and must be allowed for in order to compare the expected BTO/BFO reading errors with the best-fit BTO/BFO residuals. If this is not done, then accurate correspondence with statistical metrics is impossible. In other words, just getting a fit consistent with the expected mean and standard deviation is not very discriminating because it ignores the even larger number of statistical metrics which the True Route must also satisfy. The question is, how can we remove the systematic errors in the handshake location predictions of the flight model (using GDAS data) so that the best-fit residuals are essentially just the reading errors?
I have found that this is indeed possible if the systematic prediction errors are small. Basically, the procedure is to fit lat/lon positions at the handshake times. I use only the five handshake points from 19:41-00:11. These data can be well-fitted with a single set of seven route parameters with no maneuvers, and this eliminates the need to make any assumptions about the FMT except that it occurred prior to 19:41. The MH370 data recorded by Inmarsat between 19:41 and 00:11 UTC on 7-8 March 2014 is entirely consistent with a path during that period, while southbound and heading into the Southern Indian Ocean, without major maneuvers such as turns or holds or major speed changes. With the assumption of a preset route during this period using the auto-pilot, solutions may be found consistent with the BTO and BFO data, comprising 5 sets of BTOs and BFOs at 5 known handshake times. The 23:14 phone call BFOs are not used because there is no independent determination of that channel’s frequency bias (offset).
Once I have the five fitted locations, I compare them to the model-predicted locations to generate a set of five along-track position errors and five cross-track position errors. Then I find the ground speed errors for each route leg that produce the along-track position errors. I also find the lateral navigation errors that would produce the cross-track position errors. Note that the cross-track position errors do not occur for LNAV (great circle) routes, but only for constant track and constant heading routes. The ground speed errors are caused by airspeed errors and by GDAS along-track wind errors. The airspeed errors are small (probably 1 kt or less) and are caused primarily by errors in the GDAS temperature data. The along-track wind error is probably at least several kts at any given location and time. However, we only need the average wind error for each ~1-hour long leg along the route, and these leg average errors will generally be smaller. I estimate that the overall along-track ground speed error using my model is about 1-2 kts. That means that we must constrain the fit so that the difference between the model-predicted location and the best-fit location (in the along-track direction) is equivalent to a maximum ground speed error less than about 1-2 kts. I also expect the ground speed errors to vary smoothly with time and location. So, I put constraints on the GSEs in terms of peak value and smoothness during the route fitting process.
For the constant track and constant heading navigation modes, the cross-track position errors are caused by a combination of cross-track GDAS wind errors and FMC lateral navigation bearing errors. The lateral navigation errors are largely undocumented publicly, but I expect them to be a small fraction of a degree for the constant track modes. The lateral navigation errors in the constant heading modes are caused primarily by errors in the predicted cross-track winds, and they will be larger than the lateral navigation errors in the constant track modes. In LNAV there are no cross-track position errors.
One benefit of this route fitting method is that it requires a very good, but not a perfect model. It must predict the locations, based on the 7 route parameters and the GDAS 4-D wind and temperature data, with an accuracy of about 5-10 NM. This is a significantly larger region than the location uncertainty (in one dimension) due to BTO reading noise. Still, one can estimate the true locations with the full precision of the satellite data following the route fitting method described above, using the statistical metrics to separate the truly random portion of the residuals from the systematic, non-random model/GDAS errors. When the synthesized locations are the True Route, the best-fit residuals will behave statistically identically with the BTO/BFO reading errors.
It is also possible to compute a single figure of merit for a given fit based on the 10-12 metrics described above. I use Fisher’s chi-squared combination calculation that finds a single percentile value that is most consistent with the percentile values for each of the independent statistics being combined. The percentile values represent the percentage of random trials which are no better than this fit result, assuming the null hypothesis. In the MH370 case the null hypothesis is the same for each statistic – that the fitted route is the True Route. The alternative hypothesis is that the route is not True for at least one of the statistics. The expected value of all individual and combined percentiles is 50%; that is, half of the random trials will fit better, and half will fit worse. We can use no other assumption than the route is True, because we have no idea what the values of the metrics should be when the route is not True. We only know their expected values (and their standard deviations) for the case when the route is True. Thus, we expect the True Route will have an expected Fisher combined percentile value of 50% (with a standard deviation of 29%). Non-True (i.e., incorrect) routes will have Fisher percentile values significantly less than 50%. This is how incorrect routes are discriminated. The route fitter objective function maximizes the Fisher percentile, so each trial is trying to match the expected values of the statistics for the True Route.
Two related activities have been carried out to validate the percentile calculations. First, a simulated processor was coded which generated random BTO and BFO reading errors. Then all the individual statistics and their percentile values were computed. Finally, the combined Fisher percentile was computed. Using 100,000 trials, the actual percentile values were found to be within 1% of their nominal values for all the individual and combined statistics. In order to achieve this result, the statistics were segregated into four classes, each with is own Z statistic and probability density function. The second validation experiment involved injecting random BTO and BFO reading errors into the route fitter program itself and verifying the correct combined percentiles were obtained.
It will be appreciated that this statistical fitting method, with a dozen or so metrics and with synthesized handshake locations, involves considerable computational time, so it is not ideal for identifying Regions of Interest (ROI). The previous generation of route fitters using just BTO/BFO metrics is much more efficient for evaluating the very large number of combinations of route parameters in order to identify all ROI, which may then be refined and evaluated using the statistical fitting method for assessment and comparison.
So far, after a lengthy systematic search, we have identified quite a few ROI, and these are being evaluated with the new statistical method. I have already demonstrated that one route in particular (LNAV at 180 degrees true bearing through BEDAX at LRC and at FL390, ending near 34.3S) is fully consistent with the True Route using the statistical method. At the present time, numerous other ROI are being evaluated and compared. Preliminary results to date show them to be inferior fits, but this work is not yet completed.
The goal of this work was to develop a means to better discriminate among routes by using additional metrics, including PDA and numerous correlation coefficients, and to compensate for systematic errors in model-predicted locations so that the best-fit residuals may be directly compared with the known BTO/BFO reading errors. Our initial results are promising. It may actually be possible to demonstrate that there is only one route solution which is fully consistent with the satellite data. More complete and detailed information will be provided once our assessments are finished.