How Should Store-and-Forward Really Be Designed?
In engineering monitoring projects, there is a seemingly simple but actually important question:
Should an RTU buffer data, or store data?
Product specifications often use terms such as:
- Data buffering
- Local data storage
- Historical data
- Offline retransmission
- Store-and-Forward
These terms are closely related, but they do not solve exactly the same problem.
This distinction becomes especially important in dam safety, tailings dams, slope monitoring, bridge monitoring, tunnels, mining, hydrology, and other remote monitoring applications, where communication interruptions are not unusual.
For example, what happens if the 4G connection suddenly goes down for 30 minutes?
Sensor
↓
RTU
↓
4G Network ×
↓
Cloud Platform
What happens to those 30 minutes of monitoring data?
If the RTU has no reliable local data mechanism, the data may simply be lost.
With a properly designed Store-and-Forward mechanism, the architecture becomes:
Network Available
Sensor → RTU ─────────────────────→ Cloud Platform
│
│ Network Failure
↓
Local Storage
│
│ Network Recovery
↓
Historical Data
↓
Ordered Retransmission
↓
Cloud Platform
This is the real purpose of data continuity design.
1. Data Buffering ≠ Data Storage
A simple way to understand the difference is:
| Data Buffering | Historical Data Storage | |
| Primary purpose | Temporarily hold data before transmission | Preserve historical records |
| Typical scenario | Data collected and waiting to be transmitted | Maintaining historical monitoring records |
| Time scale | Usually relatively short | Potentially much longer |
| Must be persistent? | Not necessarily | Generally yes |
| After network recovery | Data is typically sent | Data can be queried, exported, or retransmitted |
| Typical function | Multiple Acquisition → Centralized Reporting | Historical Logging |
| Relationship to Store-and-Forward | One possible implementation mechanism | One of the foundations for reliable retransmission |
So:
Buffering answers “Where do we temporarily hold the data?” while storage answers “How long should the data be retained, why is it retained, and how will it be used?”
Confusing these concepts can lead to important design problems.
2. What Exactly Is Data Buffering?
Consider a typical engineering monitoring application.
Suppose:
- Sensors are sampled every 10 minutes
- 4G communication is available
- Data is uploaded once every hour
The RTU might operate like this:
10:00 Acquire → Buffer
10:10 Acquire → Buffer
10:20 Acquire → Buffer
10:30 Acquire → Buffer
10:40 Acquire → Buffer
10:50 Acquire → Buffer
11:00
↓
Send all accumulated data
This is essentially:
Multiple Acquisitions + Centralized Reporting
In other words:
Collect multiple times, report centrally in batches.
One important benefit is reducing communication frequency and therefore potentially reducing communication power consumption.
This can be particularly useful for remote monitoring sites powered by batteries or solar systems.
The VW-4816-4G documentation explicitly supports multiple acquisitions with centralized reporting, while also supporting an acquire-and-send simultaneously mode for more immediate transmission requirements.
So in this context, “buffering” is primarily:
Temporarily holding data until the next transmission opportunity.
3. What Is the Main Limitation of a Buffer?
The key characteristic of a buffer is:
It primarily serves the transmission process.
For example:
Acquire
↓
RAM / Buffer
↓
Wait for transmission
↓
Transmit successfully
↓
Release buffer
Once the data has been successfully uploaded:
Data → Cloud Platform
↓
Buffer can be released
There may be no reason to retain that particular copy in the transmission buffer.
Therefore:
A buffer is not automatically a historical database.
This is why:
An RTU that “supports buffering” does not necessarily have a complete historical data retransmission mechanism.
4. What Is Historical Data Storage?
Historical storage has a different purpose.
It asks:
Should this monitoring record be retained as historical data?
For example:
Time: 08:00
Pressure: 125.6 kPa
Time: 08:10
Pressure: 125.9 kPa
Time: 08:20
Pressure: 126.2 kPa
These are not merely data packets waiting to be transmitted.
They are:
Historical engineering monitoring records.
Historical storage therefore needs to consider things such as:
- Timestamp
- Channel ID
- Measurement value
- Engineering value
- Status
- Trigger condition
- Data integrity
- Storage capacity
- Data retrieval
- Data export
- Retransmission after network recovery
The VW-4816-4G documentation describes Historical Logging as automatically collecting and storing process variables in internal non-volatile memory according to preset intervals or trigger conditions, allowing the device to maintain local records independently of real-time communication.
That is fundamentally different from a simple transmission buffer.
5. Why Does Engineering Monitoring Need Historical Storage?
Because engineering monitoring has a special characteristic:
Past data can be just as important as current data.
Suppose a slope monitoring system detects an anomaly.
The platform shows:
14:00 Normal
14:10 Normal
14:20 Abnormal
14:30 Abnormal
Engineers may not only want to know:
“Is the system abnormal now?”
They may also want to know:
“When did the anomaly actually begin?”
They may then need to analyze the relationship between:
Displacement
Pore Pressure
Water Level
Rainfall
Temperature
over time.
Without historical data, many of these analyses become difficult or impossible.
6. What Exactly Is Store-and-Forward?
Store-and-Forward can be understood very simply as:
Store it first, then forward it later.
It addresses a very specific problem:
Communication is temporarily unavailable, but field data must not be lost.
For example:
08:00 Acquire → Upload successfully
08:10 Acquire → Upload successfully
08:20 Acquire → 4G connection lost
08:30 Acquire → Store locally
08:40 Acquire → Store locally
08:50 Acquire → Store locally
09:00 Acquire → Store locally
09:10 4G restored
A properly designed system should then do something like:
09:10 Network restored
↓
Check unuploaded data
↓
08:20
08:30
08:40
08:50
09:00
↓
Retransmit in chronological order
↓
Cloud Platform
The VW-4816-4G documentation defines Offline Retransmission, also called Store-and-Forward or data backfill, as storing collected data locally when communication is interrupted and automatically uploading the missing historical data in order once communication is restored.
7. Real Store-and-Forward Is More Than “Store Something”
This is an important point when evaluating an engineering RTU.
A product may claim:
“Supports offline data storage.”
But that statement alone is not enough.
A complete Store-and-Forward mechanism should consider several stages:
① Data Acquisition
↓
② Local Timestamp
↓
③ Persistent Local Storage
↓
④ Transmission Attempt
↓
⑤ Check Transmission Status
↓
┌───┴───┐
↓ ↓
Success Failure
↓ ↓
Confirm Retain
↓
Wait for Recovery
↓
Retry Transmission
The critical question is:
When can the system consider a data record truly safe?
8. One of the Most Important Questions: How Is Transmission Confirmed?
Suppose:
RTU → 4G → Cloud Platform
The RTU sends a data packet.
But something happens during transmission:
RTU has transmitted
↓
Network connection drops
↓
Did the cloud platform actually receive it?
This creates a classic problem:
Transmission does not necessarily mean successful delivery.
Therefore, a reliable Store-and-Forward architecture should ideally follow a process such as:
Data Generated
↓
Local Storage
↓
Transmit
↓
Confirmation
↓
Confirmed Successfully
↓
Mark as Uploaded
Rather than simply:
Transmit
↓
Delete
Otherwise, communication failures can result in data loss.
9. Why Is Chronological Order Important?
Engineering monitoring data is inherently time-series data.
For example:
10:00 101.2
10:10 101.4
10:20 101.7
10:30 102.1
10:40 102.8
If the network recovers and the RTU retransmits the data in this order:
10:40
10:10
10:30
10:20
10:00
the data may technically still exist, but the order is not ideal for many monitoring platforms, trend displays, and event analyses.
Therefore:
Historical data should generally be retransmitted according to its original acquisition sequence.
The VW-4816-4G documentation specifically describes uploading the missing historical data in order after communication recovery.
10. Another Easily Overlooked Issue: Timestamps
Suppose the RTU collects data while the network is down:
10:00
10:10
10:20
10:30
The network does not recover until:
11:00
The platform should not treat all these records as:
11:00
Instead:
Measurement Time ≠ Transmission Time
This distinction is extremely important in remote engineering monitoring.
The original measurement timestamp should remain associated with the measurement, even when the data is transmitted much later.
11. Buffering and Historical Storage Can Form a Two-Layer Architecture
A practical architecture can be understood like this:
RTU
│
Sensor Data
│
↓
Edge Processing / Logic
│
┌────────┴────────┐
↓ ↓
Data Buffer Historical Storage
│ │
↓ ↓
Waiting for Send Local Records
│ │
└────────┬────────┘
↓
4G Network
↓
Cloud / SCADA
Here:
Buffering answers:
“When should this data be transmitted?”
Historical storage answers:
“Should this data remain available as a historical record?”
Store-and-Forward answers:
“What happens when the communication path is temporarily unavailable?”
These are related, but they are different layers of the system.
12. How Long Should the RTU Store Data?
This is one of the most important design parameters.
Suppose:
- 16 channels
- One acquisition every 10 minutes
- One complete record per acquisition
Then approximately:
24 × 6 = 144 records/day
For 30 days:
144 × 30 = 4,320 records
If the sampling interval is reduced to one minute:
24 × 60 = 1,440 records/day
For 30 days:
43,200 records
Therefore:
Storage capacity cannot be evaluated separately from sampling interval, channel count, record structure, and required retention period.
Instead of asking only:
“How many megabytes does the RTU have?”
A more useful question is:
“Under my actual monitoring configuration, how many days of data can it retain?”
13. How Long Can the Communication Outage Last?
This is another critical Store-and-Forward design parameter.
Scenario A: Occasional short interruptions
Network lost for 5 minutes
↓
Local buffering
↓
Network recovery
↓
Automatic retransmission
A relatively simple buffering mechanism may be sufficient.
Scenario B: 12-hour communication outage
12 hours without network
↓
Continuous acquisition
↓
Local storage
↓
Network recovery
↓
Large amount of historical data
↓
Retransmission
This requires more than a simple transmission buffer.
It requires:
Persistent historical storage + Store-and-Forward.
Scenario C: Several days without communication
Now the system must also consider:
- Storage capacity
- Data retention policy
- Oldest/newest record management
- Retransmission strategy
- Data integrity
- Power consumption
- Transmission speed after recovery
This is where Store-and-Forward becomes a genuine engineering design issue.
14. What Happens If an Alarm Occurs During a Network Outage?
This is another important consideration.
Suppose normal operation is:
Acquire every 10 minutes
Report every 30 minutes
Then suddenly:
Pressure exceeds threshold
If the RTU simply waits for the next scheduled report:
Wait for next scheduled transmission
the alarm may be delayed.
A more responsive strategy is:
Normal Operation
↓
Scheduled Acquisition
↓
Normal Reporting
↓
Abnormal Condition
↓
Threshold Trigger
↓
High-Priority Data
↓
Immediate Transmission Attempt
The VW-4816-4G supports independent threshold monitoring for acquisition channels. When a preset threshold condition is triggered, the device can interrupt its regular reporting cycle, switch to high-priority alarm reporting, and upload the abnormal data packet immediately.
15. What If the Network Is Down Exactly When the Alarm Occurs?
This is where Store-and-Forward becomes particularly valuable.
For example:
14:00 Normal
14:10 Normal
14:20 Abnormal!
↓
Network Down
↓
Alarm cannot be transmitted
↓
Alarm data stored locally
↓
14:30 Abnormal data stored
14:40 Network restored
↓
Historical data processed
↓
14:20, 14:30 … retransmitted
This creates:
Alarm + Store-and-Forward
rather than simply:
Alarm + Network
The two approaches have very different reliability characteristics.
16. Buffering Can Also Support Low-Power Design
Engineering monitoring RTUs are often deployed without mains power.
Typical locations include:
- Remote mountains
- Dams
- Slopes
- Tailings facilities
- Hydrological monitoring sites
The RTU may operate like this:
Sleep
↓
Wake Up
↓
Acquire
↓
Calculate
↓
Buffer
↓
Communicate
↓
Sleep Again
If the device establishes a communication session after every individual acquisition, communication power consumption may increase.
An alternative strategy is:
Multiple Acquisitions
↓
Local Buffer
↓
Centralized Transmission
↓
Sleep
The VW-4816-4G supports both multiple acquisitions with centralized reporting and acquire-and-send simultaneously, together with low-power operation and intelligent power management for external sensors.
Therefore:
Buffering can be both a data reliability mechanism and part of a low-power communication strategy.
17. Ten Questions to Ask When Evaluating Store-and-Forward
When selecting an engineering monitoring RTU, I would not stop at:
“Does it support Store-and-Forward?”
Instead, ask:
1. Where is the data stored during a network outage?
RAM?
Flash?
Other non-volatile memory?
2. Is retransmission automatic after network recovery?
Or does it require manual intervention?
3. Is historical data retransmitted chronologically?
4. Does each record retain its original acquisition timestamp?
5. How does the RTU determine that a record has been successfully delivered?
6. What happens when retransmission fails?
Transmission fails
↓
Data retained
↓
Retry later
Or is the data discarded?
7. How long can the device retain data?
Don’t look only at storage capacity in MB.
Convert it into:
Actual retention time under the intended monitoring configuration.
8. What happens when the storage becomes full?
Stop acquisition?
Overwrite oldest data?
Generate an alarm?
This should be clearly defined.
9. Are alarm records handled differently from normal records?
In many applications, they should be.
10. What happens to real-time data while historical data is being retransmitted?
This is often overlooked.
18. A More Complete RTU Data Architecture
Putting everything together:
Sensors
│
↓
┌──────────────┐
│ RTU │
│ │
│ Data Acquire │
│ ↓ │
│ Edge Compute │
│ ↓ │
│ State Logic │
│ ↓ │
│ Threshold / │
│ Event Logic │
└──────┬───────┘
│
┌────────┴────────┐
↓ ↓
Data Buffer Historical Storage
│ │
↓ ↓
Waiting to Send Local Records
│ │
└────────┬────────┘
↓
Store-and-Forward
↓
4G
↓
Cloud / SCADA
This architecture addresses:
Acquisition → Calculation → Decision → Buffering → Storage → Communication → Retransmission
rather than simply:
Sensor → 4G → Cloud
19. Why Is the VW-4816-4G Relevant to This Architecture?
The VW-4816-4G is not simply a vibrating wire data transmitter.
Its documentation describes a combination of:
- Multi-channel vibrating wire acquisition
- RS485/Modbus hybrid acquisition
- Edge Computing
- Engineering-value conversion
- Historical Logging
- Offline Retransmission / Store-and-Forward
- Threshold monitoring
- High-priority alarm reporting
- Multiple Acquisitions + Centralized Reporting
- Acquire-and-Send
- 4G communication
- MQTT, Modbus, SL651, HJ212 and private protocols
This positions the device as an engineering monitoring RTU combining:
Data acquisition + edge computing + local storage + alarm logic + remote communication
The product documentation specifically describes Offline Retransmission and Historical Logging as separate functions.
That distinction is important.
20. The Real Question Is Not “Does It Have a Buffer?”
The better way to think about RTU data handling is:
Data Lifecycle Management
In other words:
Data Generated
↓
Data Acquired
↓
Edge Processing
↓
Local Buffer
↓
Persistent Storage
↓
Transmission
↓
Delivery Confirmation
↓
Successfully Archived
↓
If Failed → Retain
↓
Network Recovery
↓
Historical Retransmission
↓
Cloud Database
This is a much more complete engineering approach.
21. A Simple Checklist to Remember
When a monitoring RTU claims:
“Supports Store-and-Forward”
don’t stop there.
Ask:
What exactly is stored?
What exactly is forwarded?
Where is it stored?
How long can it be retained?
When is it deleted?
How is successful delivery confirmed?
What happens if retransmission fails?
Are records retransmitted chronologically?
How are alarm records handled?
What happens to real-time data during historical backfill?
If these questions can be answered clearly, then the Store-and-Forward function has genuine engineering value.
Conclusion: Reliable Remote Monitoring Should Not Assume That the Network Is Always Available
For ordinary IoT applications, a short communication interruption may simply result in a small amount of missing data.
For applications such as:
- Dam safety
- Tailings dams
- Slope monitoring
- Bridge monitoring
- Tunnels
- Mining
- Hydrological monitoring
data continuity can be important for subsequent engineering analysis and anomaly investigation.
Therefore, a mature engineering monitoring RTU should not be designed around the assumption:
“The 4G network will always be available.”
Instead, the system should assume:
“Communication will eventually be interrupted.”
And then design for:
Local Storage + Buffering + Store-and-Forward + Timestamping + Retry + Alarm Priority
The objective is not simply to:
“Store data when the network is down.”
The objective is to ensure that:
Even when communication temporarily disappears, field measurements continue to be acquired and preserved, and the historical data can return to the monitoring platform in an orderly manner when communication is restored.
That is what a reliable engineering monitoring RTU should be designed to achieve.
Next article
Why Do Engineering Monitoring RTUs Need “Event-Driven” Operation?
From Scheduled Acquisition to Active Response to Abnormal Conditions
The next step in this series can connect:
Scheduled Acquisition → Data Buffering → Store-and-Forward → Threshold Trigger → Event Trigger → Increased Sampling → High-Priority Reporting → Automatic Return to Normal Operation
This takes the discussion from “What an RTU is” to “How an RTU should actually operate in the field.”
#EngineeringMonitoring #RTU #StoreAndForward #DataLogger #VibratingWire #GeotechnicalMonitoring #StructuralHealthMonitoring #DamSafety #TailingsDam #SlopeMonitoring #RemoteMonitoring #EdgeComputing #IndustrialIoT #SmartInfrastructure
