From Local Data Storage to Remote Platform Reliability
During foundation pit construction, monitoring systems need to continuously collect data on retaining structure deformation, support axial forces, groundwater levels, pore water pressure, and surrounding ground settlement.
In an automated monitoring system, field devices typically transmit collected data to a remote platform through 4G, Ethernet, or other communication networks. Engineers can then review monitoring curves, analyze trends, and determine whether further inspections or corrective actions are necessary.
However, an important question is often overlooked:
What happens when field equipment continues collecting data, but a network outage prevents that data from reaching the remote platform?
If the acquisition device only supports real-time transmission without reliable local storage, even a temporary communication failure may result in missing monitoring records.
For foundation pits, slopes, dams, tunnels, and other engineering monitoring applications, store-and-forward functionality is therefore an important mechanism for improving data continuity.
This article explains how store-and-forward works, why local storage matters, and what engineers should consider when designing a reliable remote monitoring system.
1. Why Do Foundation Pit Monitoring Systems Experience Network Outages?
Many engineers assume that using 4G communication solves the connectivity problem. In reality, 4G is a communication method, not a guarantee of uninterrupted connectivity.
Several factors can cause communication interruptions at construction sites.
1.1 Unstable Signals in Challenging Construction Environments
Foundation pits are often located in densely built urban areas. Nearby high-rise buildings, underground spaces, temporary structures, and heavy construction equipment can affect wireless communication.
As excavation progresses, the communication conditions between field devices and cellular base stations may also change.
For example:
- Acquisition devices are installed underground or deep inside an excavation.
- Nearby buildings obstruct wireless signals.
- Metal structures or temporary facilities affect signal quality.
- Changes in the construction area make previously suitable installation locations less effective.
These conditions may weaken signals, increase communication delays, or cause temporary disconnections.
1.2 Cellular Networks Are Not Always Available
Even when signal strength is adequate, mobile network congestion, base station maintenance, network handovers, or local service interruptions may temporarily prevent a device from connecting to a server.
This can be particularly relevant at remote construction sites, mountain projects, and large infrastructure developments.
1.3 Device-to-Server Communication Can Fail
Not all communication failures originate from the wireless network itself.
Other potential causes include:
- Disconnected sessions between devices and servers
- Temporary server unavailability
- Incorrect communication settings
- Interrupted MQTT connections
- Interface errors or request timeouts
Therefore, checking whether a device has a 4G signal is not sufficient.
The real question is whether the monitoring data has reached the platform and been successfully stored.
2. Why Is Real-Time Transmission Alone Not Enough?
Consider a foundation pit monitoring project in which an automated acquisition device collects data every 10 minutes.
Under normal conditions, the system generates six records per hour.
If the network fails for three hours while the device continues collecting measurements, it should generate 18 records during that period.
Without local storage, or if records are retained only in volatile memory, those measurements may be lost.
When connectivity returns, the platform may resume receiving new data, but the missing historical records cannot automatically be recovered.
This creates three major problems.
2.1 Gaps in Monitoring Curves
Engineers may see a period with no data on the monitoring platform.
However, the absence of records does not mean that the monitored structure remained unchanged. It may simply mean that the system could not provide the measurements for that period.
If the gap occurs during excavation, support installation, or groundwater lowering, it may affect the interpretation of important construction stages.
2.2 An Incomplete Picture of Engineering Changes
Foundation pit monitoring is not just about checking individual measurements. It also involves analyzing how values change over time.
For example, retaining wall displacement may increase continuously even when individual measurements remain below the alarm threshold.
The rate of change may still warrant further investigation.
If intermediate records are missing, engineers may find it difficult to determine when the change began, whether it accelerated, or whether it was associated with a particular construction activity.
2.3 The System Appears Online, but the Data Is Incomplete
Some monitoring systems treat device connectivity and successful data reception as if they were the same thing.
They are not.
A device can have a network connection without successfully transmitting its measurements. Similarly, data transmission may succeed at the communication level without guaranteeing that the platform has durably stored the records.
System reliability must therefore be evaluated across the entire process: acquisition, storage, transmission, reception, and database persistence.
3. What Is Store-and-Forward?
Store-and-forward is a data transmission mechanism in which field equipment continues collecting and storing data locally during communication outages. Once connectivity is restored, the device transmits historical records that have not yet been successfully delivered.
The basic process is:
Normal acquisition → Local storage → Data transmission → Network outage and continued storage → Network recovery → Historical data upload → Platform validation and storage.
Store-and-forward is more than simply resending data after a network reconnects.
A reliable implementation must determine which records have not been successfully delivered, identify records already received, and prevent duplication or incorrect ordering.
From a system design perspective, store-and-forward involves three key layers:
- Field acquisition device: Retains measurements during communication failures.
- Communication and transmission mechanism: Sends outstanding records after connectivity returns.
- Remote platform: Identifies historical records, handles duplicates, and stores data in the correct temporal sequence.
If any of these layers is poorly designed, the system may fail to achieve the expected level of data reliability.
4. How Does the Store-and-Forward Process Work?
Step 1: Collect Data According to the Configured Schedule
The acquisition device reads sensor measurements at predefined intervals.
For vibrating wire sensors, it can measure the corresponding resonant frequency and, when configured and supported, convert the readings into engineering values using the appropriate sensor coefficients.
For compatible digital instruments, the device may read measurements through the supported communication interface.
Each record should contain the necessary identification and measurement information, such as:
- Device ID
- Sensor or channel ID
- Actual acquisition timestamp
- Measurement value
- Data quality or status information
- Relevant configuration or version information
The acquisition timestamp is particularly important. The platform needs to know when the measurement was actually taken, not merely when it arrived at the server.
Step 2: Save Data Locally Before or Alongside Transmission
For monitoring projects that require data continuity, a sound design is to write acquired records to reliable local storage before attempting transmission, or to use an equivalent mechanism that protects records until delivery is confirmed.
This helps ensure that a failed upload does not immediately result in data loss.
Local storage technologies vary according to hardware design, power consumption, capacity, and reliability requirements.
Engineers should consider:
- Whether records survive an unexpected power interruption
- How the device behaves when storage is full
- Whether the storage medium is suitable for long-term field operation
- How interrupted write operations are recovered
- How historical records can be retrieved
A product specification that lists “local storage” does not, by itself, guarantee that data can never be lost.
Step 3: Identify Failed Uploads and Retain Pending Records
When the device cannot connect to the server, or a transmission does not receive the expected confirmation, the relevant records should remain available for subsequent delivery.
It is useful to distinguish between several states:
- Data has been acquired.
- Data has been written to local storage.
- Data has been transmitted.
- The server has received the data.
- The platform has successfully persisted the data in its database.
These states are not necessarily equivalent.
For example, a device may send a record but lose its connection before receiving confirmation. It may not know whether the server received the record.
Deleting the local record immediately could cause data loss. Resending it without appropriate duplicate handling could create duplicate entries.
A reliable implementation therefore needs suitable acknowledgement, retry, and record-identification mechanisms.
Step 4: Automatically Upload Historical Data After Network Recovery
Once communication is restored, the device can transmit records that remain pending.
Common strategies include:
- Uploading historical records before resuming routine uploads
- Scheduling historical and real-time traffic separately
- Limiting the rate of historical uploads to protect real-time communication
- Sending records in batches based on timestamps or sequence numbers
- Retrying failed transmissions according to a defined policy
The appropriate strategy depends on the project’s requirements for timeliness, data completeness, and available bandwidth.
For safety-sensitive monitoring applications, the goal is not simply to upload all historical data as quickly as possible. Current measurements and important alarms must also remain timely.
Step 5: Validate and Store Records on the Remote Platform
When historical data arrives, the platform needs to identify existing records and organize the new records according to their actual acquisition times.
Important checks include:
- Duplicate records
- Missing time intervals
- Invalid timestamps
- Incorrect device or channel identifiers
- Identification of historical uploads
- Recalculation or updating of derived statistics when necessary
Records can be identified using a suitable combination of device ID, channel ID, acquisition timestamp, sequence number, or another unique identifier.
The exact approach depends on the acquisition interval, timestamp precision, and data model. A timestamp alone should not automatically be assumed to uniquely identify every measurement.
5. How Much Local Storage Capacity Is Required?
Local storage capacity is a key parameter in store-and-forward design.
If a device can retain only a few hours of data, an extended network outage may still cause records to be lost.
A simple estimation formula is:
Required storage ≈ Record size × Records per day × Supported outage duration × Safety factor
Consider a monitoring node that generates one record every 10 minutes:
- Records per day: 144
- Assumed size per record: 200 bytes
- Target outage duration: 7 days
The raw record content would require:
144 × 200 × 7 = 201,600 bytes, or approximately 197 KiB.
This is only an illustrative estimate. Actual storage requirements may be significantly higher when each record includes multiple channels, temperature readings, status fields, checksums, and other metadata.
System design should also account for indexes, file-system overhead, logs, database structures, and a suitable safety margin.
For a multi-channel device, calculations should use the actual record-generation model for the entire unit rather than estimating storage for only one sensor.
The storage policy also matters:
- Will the device overwrite the oldest records or stop writing when storage is full?
- Which historical records, if any, may be deleted?
- When can records be safely cleared after successful upload?
- Can the system issue a warning when storage approaches capacity?
- What happens if the network remains unavailable for an extended period?
For important monitoring data, local records should not be deleted before the system has adequate confirmation that the data has been reliably stored on the platform.
6. Why Does Successful Data Recovery Not Guarantee a Fully Reliable System?
Store-and-forward addresses data continuity during communication interruptions. It does not replace a complete data reliability strategy.
Several additional issues deserve attention.
6.1 Timestamps Must Be Trustworthy
Even if every record is successfully uploaded, inaccurate device clocks can prevent the platform from reconstructing the correct sequence of engineering events.
For example, a device may lose its time reference after a restart, or multiple acquisition units may have significantly different clock settings.
A monitoring system should therefore implement appropriate time synchronization, clock anomaly detection, and timestamp management.
For critical applications, the system should also define how records are handled when an accurate timestamp is unavailable. Uncertain timestamps should not be presented as precise measurements of time.
6.2 Duplicate Records Must Be Prevented
Suppose a device sends a record but loses communication before receiving the server’s acknowledgement.
After reconnecting, it sends the same record again. The platform may receive that record twice.
Without suitable duplicate handling, this can distort statistics and produce misleading monitoring curves.
Store-and-forward systems should therefore use appropriate record identifiers, acknowledgement mechanisms, and idempotent processing so that repeated delivery does not create multiple independent measurements.
6.3 Real Engineering Changes Must Be Distinguished from Data Anomalies
A sudden change in a monitoring curve does not automatically mean that the structure has experienced an abnormal physical change.
Potential causes include actual engineering movement, sensor faults, wiring problems, communication errors, incorrect configuration, or data-processing issues.
Likewise, successful receipt of historical data does not guarantee that the original measurement was valid.
Reliable monitoring systems should consider data quality indicators, device status, sensor characteristics, and engineering context together.
6.4 Remote Alarms May Not Be Delivered During an Outage
This is an important issue that is often overlooked.
Suppose a field device detects a measurement exceeding a configured threshold while the network is unavailable.
Even if the device stores the measurement locally, the remote platform may not receive the alarm immediately unless the system has another functioning communication path or an independent local alarm mechanism.
When the network returns, the platform may receive the historical abnormal measurement, but the opportunity for immediate notification may already have passed.
Engineers should therefore determine:
- Whether threshold checks run on the device or on the platform
- Whether the device can perform local checks during an outage
- Whether audible, visual, or other on-site alarms are available
- Whether a backup communication path exists
- Whether historical abnormal measurements generate retrospective notifications
- How real-time alarms are distinguished from historical event notifications
Store-and-forward improves data completeness, but it cannot independently guarantee real-time remote alarms during a network outage.
6.5 Local Storage Cannot Support Unlimited Outages
If communication remains unavailable for too long, local storage may eventually fill up.
The system therefore needs storage-capacity monitoring, fault notifications, and a clearly defined data-retention policy.
For important projects, engineers should also establish inspection procedures, manual data export options, or other contingency arrangements.
A reliable design does not assume that the network will never fail. It defines what the system should do when an outage exceeds expectations.
7. How Can Real-Time Monitoring and Historical Data Recovery Work Together?
In foundation pit monitoring, both historical data completeness and timely response to abnormal conditions are important, but they serve different purposes.
Historical data recovery asks whether measurements already collected can eventually reach the platform.
Real-time monitoring asks whether current changes can be detected and communicated within the required time.
A practical architecture can address these requirements at three levels.
At the device level:
- Acquire measurements according to the configured schedule.
- Store measurement records reliably.
- Perform local threshold checks when supported.
- Monitor storage capacity and device status.
- Transmit pending records after connectivity returns.
At the communication level:
- Detect connection failures.
- Retry unsuccessful transmissions.
- Manage the priority of historical and real-time traffic.
- Avoid allowing historical uploads to interfere with critical real-time information.
At the platform level:
- Receive real-time and historical data.
- Identify duplicate records.
- Validate timestamps and device information.
- Detect data gaps and offline devices.
- Apply appropriate alarm rules to current and historical events.
Where strict response times are required, engineers should also evaluate backup communication paths, local alarms, and clearly defined response procedures.
These capabilities must be verified for the actual equipment and system configuration. They should not be assumed to be available on every RTU.
8. How to Design a More Reliable Data Path for Foundation Pit Monitoring
Consider a foundation pit project using vibrating wire sensors to monitor support forces and groundwater-related parameters, with compatible digital instruments deployed at selected locations.
A possible system architecture consists of six layers:
- Sensor layer: Measures retaining structure behavior, support forces, and groundwater-related parameters.
- Field acquisition layer: Uses vibrating wire acquisition equipment or a compatible hybrid acquisition RTU to collect sensor readings.
- Local storage layer: Retains measurement records, timestamps, and relevant device status information.
- Communication layer: Transmits data through 4G or another suitable network.
- Platform layer: Provides data storage, visualization, trend analysis, and alarm management.
- Operations and maintenance layer: Monitors offline devices, storage capacity, communication failures, and the need for field inspection.
With this architecture, field equipment continues collecting and storing data during an outage. After communication is restored, it transmits pending records, and the platform identifies, validates, and stores them.
A suitable multi-channel vibrating wire hybrid acquisition RTU may also integrate vibrating wire measurements, communication with compatible RS485 digital instruments, 4G transmission, and local data processing, depending on the model and configuration.
However, the channel count, protocol support, storage capacity, and alarm functions vary between devices. System design should be based on verified product specifications and field testing.
A hybrid RTU is not a universal replacement for every instrument in a monitoring system. Automated surveying equipment, specialized inclinometers, and other instruments may still require dedicated interfaces or acquisition methods.
9. How Can You Verify That Store-and-Forward Actually Works?
A product specification stating “store-and-forward supported” does not necessarily mean that the equipment meets every reliability requirement of a particular project.
During equipment evaluation or project acceptance, consider the following tests.
Test 1: Simulate a Network Outage
Disconnect the communication link while the device is collecting data.
Confirm that acquisition continues and records are retained locally. Check whether the number of stored records matches the expected acquisition schedule.
Test 2: Restore Connectivity
Reconnect the network and observe whether the device resumes transmission automatically and uploads the historical records.
Check whether manual intervention is required and whether failed uploads are retried.
Test 3: Verify Data Completeness
Compare field records with platform records.
Check for missing measurements, duplicates, incorrect timestamps, incorrect ordering, and unexpected changes in values.
A platform message indicating “upload successful” is not sufficient evidence that the test has passed.
Test 4: Simulate a Device Restart
Following the equipment’s operating procedures, simulate a restart or brief power interruption.
Check whether previously saved records remain available.
This helps verify the local storage mechanism rather than merely the ability to reconnect to a network.
Test 5: Simulate an Extended Outage
Simulate a communication interruption that reflects the project’s requirements.
Confirm that storage capacity is sufficient and determine how the device behaves when storage approaches its limit.
Test 6: Verify Historical Event and Alarm Handling
In a test environment, simulate an abnormal measurement and determine how the device and platform process it.
In particular, verify that an abnormal event occurring during an outage is correctly identified as a historical event after reconnection, rather than being incorrectly treated as a newly occurring real-time event.
These tests provide a more meaningful assessment of data reliability than simply checking whether a device advertises store-and-forward functionality.
10. Conclusion: A Reliable Monitoring System Must Be Designed for Network Failure
The value of a foundation pit monitoring system lies not simply in transmitting sensor readings to a cloud platform. It lies in maintaining the most complete and trustworthy record of engineering conditions possible and providing engineers with useful information for decision-making.
4G improves remote connectivity, but it cannot guarantee that a network will always remain available.
Local storage helps reduce the risk of losing measurements during temporary communication failures. Store-and-forward allows historical records to be delivered after connectivity is restored.
Timestamp management, duplicate handling, storage-capacity monitoring, device-status checks, and alarm design are equally important parts of a reliable system.
When selecting data acquisition equipment for foundation pits, slopes, dams, and other engineering monitoring projects, engineers should look beyond measurement accuracy, channel count, and communication interfaces.
They should also ask:
- Can the device continue acquiring data during an outage?
- How long can it retain data locally?
- Does it automatically upload historical records after network recovery?
- How does it prevent missing or duplicate records?
- What happens if an abnormal condition occurs during an outage?
- How can these functions be verified under realistic field conditions?
These questions can be more important than simply comparing equipment prices.
A truly reliable monitoring system is not one that never experiences failures. It is one that protects data integrity as far as practicable and clearly indicates what happened when communication, equipment, or site conditions change.
At Hu Entropy Tech, we develop engineering monitoring data acquisition equipment, vibrating wire data loggers, multi-channel RTUs, and Industrial IoT solutions to support reliable data acquisition and remote monitoring.
Learn more about our products and solutions, please feel free to reach out.
#FoundationPitMonitoring #GeotechnicalMonitoring #VibratingWire #DataAcquisition #RTU #IndustrialIoT #StructuralHealthMonitoring #ExcavationMonitoring #EngineeringMonitoring
