The four nationwide wireless carriers have proposed a z-axis, or vertical, accuracy standard for indoor 911 location accuracy, although they stressed shortcomings of testing that has occurred and stressed that “further testing is needed to validate and confirm performance expectations of Z-axis solutions for live wireless 9-1-1 calling environments.”

The carriers proposed “a Z-axis metric of +/- 5 meters for 80% of fixes from mobile devices capable of delivering barometric pressure sensor-based altitude estimates.”

The z-axis accuracy standard proposal from the carriers was required by a 911 location accuracy order adopted by the FCC in 2015 (TR Daily, Jan. 29, 2015). The FCC will now consider what standard to adopt.

In the order, the FCC said carriers will have to deploy either dispatchable location or z-axis technology in the 25 most populous cellular market areas (CMAs) within six years, and will have to deploy either dispatchable location or z-axis technology in the 50 most populous CMAs within eight years. Non-nationwide carriers that serve these markets will get an additional year to deploy.

The carriers have worked since 2015 to conduct testing of 911 location accuracy technologies through a test bed administered by a nonprofit entity, 9-1-1 Location Technologies Test Bed LLC, established by CTIA on behalf of the carriers. Vertical location solutions were tested in Stage Z of the testing. The solutions of NextNav LLC and Polaris Wireless, Inc., were tested in that stage.

“The results of Stage Z demonstrate that it is challenging to identify a Z-axis metric that can be consistently replicated in a live 9-1-1 calling environment with only two technology vendors participating in this round of Z-axis testing, under somewhat artificial conditions,” a 134-page report on the testing concluded. “Consistent with the FCC’s Fourth Report & Order (para. 4 and 170), the proposed Z-Axis metric must be vendor-neutral and achievable across the entirety of carrier networks within the timeframe prescribed by Commission rules. Going forward, the Test Bed can be made available to administer additional rounds of Stage Z testing for Z-axis technology vendors interested in participating.”

A cover letter submitted to the FCC with the report in PS docket 07-114 by CTIA, Verizon Communications, Inc., AT&T, Inc., T-Mobile US, Inc., and Sprint Corp. noted that “Stage Z testing was conducted in the dense urban, urban, suburban, and rural morphologies of the Atlanta, San Francisco, and Chicago regions. Testing was conducted in 48 buildings of a wide variety, using 312 test points. Testing involved 30 smartphone devices, including 12 different handset models, and produced over 100,000 location estimates (i.e., fixes). … However, as explained further below, only one vendor participated in all of the regions and morphologies. Comprehensive testing of Z-axis solutions in all regions and morphologies is necessary to confirm performance in live 9-1-1 calling environments.”

“The Report finds that the Stage Z results provide helpful data, but it also notes that significant questions remain about performance and scalability for live 9-1-1 call environments,” the filing noted. “While the Report is not intended to support a comparison of the tested solutions, each of the two vendors’ solutions stopped short of testing in a comprehensive manner. Thus, key questions remain as to how and whether the results can be replicated and deployed ubiquitously in real-world production and live 9-1-1 call environments …”

The filing added that “NextNav has not deployed its MBS [metropolitan beacon system] network in Chicago. Notably, according to recommendations from ATIS, the Chicago test region was selected for the purpose of providing a more extreme weather environment to evaluate Z-axis technologies, including fluctuating indoor-outdoor temperature and pressure differences on barometric pressure sensor-based technologies. (Report at § 1) Therefore, NextNav’s solution could not be tested with more extreme weather conditions.”

Also, “Stage Z testing on Polaris Wireless’ solution included six different device models but no iPhone,” the filing noted.

Moreover, “Stage Z results reflect that sensor bias is the dominant error source for estimating altitude,” the filing added. “The test results also showed that each individual mobile device is subject to its own unique sensor bias and thus requires individualized active barometric sensor bias calibration. Adjusting for sensor bias on an individualized device-by-device basis proved manageable when dealing with dedicated application software (apps) running on six test handsets, but this calibration capability would need to be built into live production solutions, with signaling based on standards, and retested at scale to fully assess performance. (Report at § 1) It is, therefore, unclear whether the Stage Z performance results would be replicated in a live 9-1-1 calling environment.”

“Barometer manufacturers have indicated that accuracy degrades as the sensors age. However, because older devices could not support the apps provided by NextNav and Polaris Wireless, only relatively new handsets — released more recently than mid-2016 — were tested,” the filing said. “Among other reasons, the Report recommends further testing of sensor performance with a variety of device types and functionality.”

“Although Polaris Wireless tested in Chicago in the winter, the lack of extreme weather conditions at the time of testing leaves open questions about the reliability of compensated barometric-pressure-based solutions in such conditions,” the filing added.

The filing noted that the report also “observes that, ‘to evaluate these two technologies, some artificial steps and accommodations had to be taken to produce the location estimates in the test campaign.’ (Report at § 11.3) Indeed, Stage Z testing ‘essentially perform[ed] a “proof of concept” test.’ (Report at § 11.1.1.) No actual calls were placed to produce any Z-axis fixes, and standardized 9-1-1 signaling was not used. Each vendor used its own dedicated app with ‘all permission requests allowed’ (i.e., the app had unrestricted access to the barometric pressure sensor), which is not consistent with commercial practice. (Report at §§ 8.5.1.2, 8.5.2.2) For these reasons, barometric pressure sensor-based Z-axis solutions require further development, potentially including further standardization efforts, implementation into wireless network systems and production mobile devices, and then re-testing in a production configuration to determine whether the Stage Z results can be replicated and deployed ubiquitously in real-world production and live wireless 9-1-1 call environments. Further, additional factors are likely to limit the availability of barometric pressure sensor based Z-axis solutions, including lack of support for existing mobile wireless devices.”

“Applying the results from Stage Z, CTIA on behalf of the Carriers submits a Z-axis metric of +/- 5 meters for 80% of fixes from mobile devices capable of delivering barometric pressure sensor-based altitude estimates,” the filing said. “As noted, NextNav, whose testing produced 80% of fixes at 1.8 meters or less, did not test in one of the three test regions or in any rural morphologies. Polaris Wireless, whose testing covered all three test regions and all morphologies, produced 80% of fixes at 4.8 meters or less. Stage Z testing did not demonstrate that more accurate results can be consistently achieved across all testing regions, morphologies, weather conditions, and devices by the August 3rd, 2018 filing deadline in the Fourth Report & Order.”

“In sum, the Report finds that the Stage Z results provide helpful insight to the state of barometric-pressure sensor based Z-axis technologies, but it also demonstrates that significant questions remain about performance and scalability in live wireless 9-1-1 calling environments,” the filing said. “The vendors’ accompanying statements [included in the report] to the Report diverge on key findings and, of relevance to next steps, they do not agree on the primary source of vertical error. Polaris Wireless concurs with the Report, observing that ‘[o]ne of the most significant sources of error for barometric sensor-based location solutions … is bias in the device barometric sensors.’ But NextNav disagrees, questioning the Report’s ‘assertion that the dominant source of error was sensor bias calibration’ and asserting instead that ‘[w]eather is the dominant source of altitude measurement error and variability.’ The vendors’ conflicting views regarding the results of the Stage Z testing underscore the need for additional testing to develop a more common understanding of the capabilities of existing and forthcoming Z-axis technologies.

“For example, other approaches to Z-axis location estimates are emerging that rely primarily on enhanced Wi-Fi positioning technologies, and only secondarily on compensated barometric pressure (where available),” the filing continued. “These solutions may address some of the issues inherent to barometric pressure sensor-based systems, such as backward compatibility to existing handsets. A few vendors of the enhanced Wi-Fi positioning technologies have indicated the likely availability of their solutions for testing within the next calendar year.”

The filing reiterated that “further testing is needed to validate and confirm performance expectations of Z-axis solutions for live wireless 9-1-1 calling environments. While Stage Z results provide support for a metric of +/- 5 meters for 80% of fixes, additional testing may yield results that validate adoption of a more accurate metric. CTIA and the Carriers encourage the Commission to support additional Stage Z testing within the next calendar year, covering these technologies and other approaches including enhanced Wi-Fi positioning technologies, prior to adopting a final Z-axis metric.”

In its statement on the report, NextNav complained that “in an attempt to isolate and ascribe behavior to different elements of the vendors’ location systems, the report draws certain observations and conclusions that are speculative at best and counter to the test data at worst. This is concerning because a hallmark of prior testing and reporting has been a focus on performance analysis as opposed to speculation about specific implementation choices.”

The company cited “[t]he characterization of ‘periodic background calibration’ versus ‘one-time manual calibration’ and the “assertion that the dominant source of error was sensor bias calibration rather than other system variables, including accuracy and stability of the respective weather reference networks and associated compensation algorithms.” It added, “Beyond validating overall vertical accuracy performance of vendor systems, the Test Report reveals significant positive conclusions for barometric pressure-based altitude systems.”

In its statement, Polaris “provided a commented and edited response identifying factual errors, misleading statements, and misrepresented side-by-side performance comparisons,” including purported requests by Polaris concerning testing and testing procedures.

“While there is no reason to question the Polaris Wireless performance numbers in the report for what was tested, these results do not reflect the currently available Polaris Wireless barometric sensor-based capabilities that were proposed for testing,” Polaris said. “One of the most significant sources of error for barometric sensor-based location solutions, as clearly stated in the report by the Test Bed, is bias in the device barometric sensors. Polaris Wireless proposed to include an active compensation correction model that operates in an application running in the background of the device. Based on a conversation with the Test Bed and reviewing subsequent instructions on allowable procedures provided by the Test Bed, Polaris Wireless did not enable available active sensor compensation for Stage Z testing.” —Paul Kirby, paul.kirby@wolterskluwer.com

Courtesy TRDaily