nrt.monitor package

Submodules

nrt.monitor.ccdc module

class nrt.monitor.ccdc.CCDC(trend=True, harmonic_order=2, sensitivity=3, mask=None, boundary=3, **kwargs)

Bases: BaseNrt

Monitoring using CCDC-like implementation

Implementation loosely following method described in Zhu & Woodcock 2014.

Zhu, Zhe, and Curtis E. Woodcock. 2014. “Continuous Change Detection and Classification of Land Cover Using All Available Landsat Data.” Remote Sensing of Environment 144 (March): 152–71. https://doi.org/10.1016/j.rse.2014.01.011.

mask

A 2D numpy array containing pixels that should be monitored (1) and not (0). The mask may be updated following history period stability check, and after a call to monitor following a confirmed break. Values are as follow. {0: 'Not monitored', 1: 'monitored', 2: 'Unstable history', 3: 'Confirmed break - no longer monitored'}

Type:: numpy.ndarray

trend

Indicate whether stable period fit is performed with trend or not

Type:: bool

harmonic_order

The harmonic order of the time-series regression

Type:: int

beta

3D array containing the model coefficients

Type:: np.ndarray

x

array of x coordinates

Type:: numpy.ndarray

y

array of y coordinates

Type:: numpy.ndarray

sensitivity

sensitivity of the monitoring. Lower numbers are high sensitivity. Value can’t be zero.

Type:: float

boundary

Number of consecutive observations identified as outliers to signal as disturbance

Type:: int

rmse

2D float array indicating RMSE for each pixel

Type:: np.ndarray

detection_date

2D array signalling detection date of disturbances in days since 1970-01-01

Type:: numpy.ndarray

Parameters:

mask (numpy.ndarray) – A 2D numpy array containing pixels that should be monitored marked as 1 and pixels that should be excluded (marked as 0). Typically a stable forest mask when doing forest disturbance monitoring. If no mask is supplied all pixels are considered and a mask is created following the fit() call
trend (bool) – Indicate whether stable period fit is performed with trend or not
harmonic_order (int) – The harmonic order of the time-series regression
sensitivity (float) – sensitivity of the monitoring. Lower numbers are high sensitivity. Value can’t be zero.
boundary (int) – Number of consecutive observations identified as outliers to signal as disturbance
**kwargs – Used to set internal attributes when initializing with .from_netcdf()

fit(dataarray, method='CCDC-stable', screen_outliers='CCDC_RIRLS', green=None, swir=None, scaling_factor=1, **kwargs)

Stable history model fitting

If screen outliers is required, green and swir bands must be passed.

The stability check will use the same sensitivity as is later used for detecting changes (default: 3*RMSE)

Parameters:

dataarray (xr.DataArray) – xarray Dataarray including the historic data to be fitted
method (string) – Regression to use. See _fit() for info.
screen_outliers (string) – Outlier screening to use. See _fit() for info.
green (xr.DataArray) – Green reflectance values to be used by screen_outliers.
swir (xr.DataArray) – Short wave infrared (SWIR) reflectance values to be used by screen_outliers.
scaling_factor (int) – Optional Scaling factor to be applied to green and swir. When screen_outliers is 'CCDC_RIRLS' (default for CCDC), the outlier screening algorithm expects green and swir reflectance values in the [0,1] range. EO data are often scaled and stored as integer, with a scaling factor to convert between scaled and actual reflectance values. As an example, if scaled reflectance values are in the [0,10000] range, set scaling_factor to 10000.
**kwargs – to be passed to _fit

Examples

>>> from nrt.monitor.ccdc import CCDC
>>> from nrt import data

>>> # Load and prepare test data
>>> mask = (data.romania_forest_cover_percentage() > 30).astype('int')
>>> s2_cube = data.romania_20m()

>>> s2_cube['ndvi'] = (s2_cube.B8A - s2_cube.B4) / (s2_cube.B8A + s2_cube.B4)
>>> s2_cube = s2_cube.where(s2_cube.SCL.isin([4,5,7]))
>>> cube_history = s2_cube.sel(time=slice('2015-01-01', '2018-12-31'))

>>> # Instantiate monitoring class and fit the model, including outliers screening
>>> ccdcMonitor = CCDC(trend=True, mask=mask)
>>> ccdcMonitor.fit(dataarray=cube_history.ndvi,
...                 green=cube_history.B3,
...                 swir=cube_history.B11,
...                 scaling_factor=10000)

nrt.monitor.cusum module

class nrt.monitor.cusum.CuSum(trend=True, harmonic_order=2, sensitivity=0.05, mask=None, **kwargs)

Bases: BaseNrt

Monitoring using cumulative sums (CUSUM) of residuals

Implementation following method as implemented in R package bFast.

mask

A 2D numpy array containing pixels that should be monitored (1) and not (0). The mask may be updated following history period stability check, and after a call to monitor following a confirmed break. Values are as follow. {0: 'Not monitored', 1: 'monitored', 2: 'Unstable history', 3: 'Confirmed break - no longer monitored'}

Type:: numpy.ndarray

trend

Indicate whether stable period fit is performed with trend or not

Type:: bool

harmonic_order

The harmonic order of the time-series regression

Type:: int

x

array of x coordinates

Type:: numpy.ndarray

y

array of y coordinates

Type:: numpy.ndarray

sensitivity

sensitivity of the monitoring. Lower numbers correspond to lower sensitivity. Equivalent to significance level ‘alpha’ with which the boundary is computed

Type:: float

boundary

process boundary for each time series. Calculated from alpha and length of time series.

Type:: numpy.ndarray

sigma

Standard deviation for normalized residuals in history period

Type:: numpy.ndarray

histsize

Number of non-nan observations in history period

Type:: numpy.ndarray

n

Total number of non-nan observations in time-series

Type:: numpy.ndarray

critval

Critical test value corresponding to the chosen sensitivity

Type:: float

detection_date

2D array signalling detection date of disturbances in days since 1970-01-01

Type:: numpy.ndarray

Parameters:

mask (numpy.ndarray) – A 2D numpy array containing pixels that should be monitored marked as 1 and pixels that should be excluded (marked as 0). Typically a stable forest mask when doing forest disturbance monitoring. If no mask is supplied all pixels are considered and a mask is created following the fit() call
trend (bool) – Indicate whether stable period fit is performed with trend or not
harmonic_order (int) – The harmonic order of the time-series regression
sensitivity (float) – sensitivity of the monitoring. Lower numbers correspond to lower sensitivity. Equivalent to significance level ‘alpha’ with which the boundary is computed
**kwargs – Used to set internal attributes when initializing with .from_netcdf()

fit(dataarray, method='ROC', alpha=0.05, **kwargs)

Stable history model fitting

If method 'ROC' is used for fitting, the argument alpha has to be passed.

Parameters:

dataarray (xr.DataArray) – xarray Dataarray including the historic data to be fitted
method (string) – Regression to use. See _fit() for info.
alpha (float) – Significance level for 'ROC' stable fit.
**kwargs – to be passed to _fit

nrt.monitor.ewma module

class nrt.monitor.ewma.EWMA(trend=True, harmonic_order=2, sensitivity=2, mask=None, lambda_=0.3, threshold_outlier=10, **kwargs)

Bases: BaseNrt

Monitoring using EWMA control chart

Implementation following method described in Brooks et al. 2014.

Parameters:

mask (numpy.ndarray) – A 2D numpy array containing pixels that should be monitored marked as 1 and pixels that should be excluded (marked as 0). Typically a stable forest mask when doing forest disturbance monitoring. If no mask is supplied all pixels are considered and a mask is created following the fit() call
trend (bool) – Indicate whether stable period fit is performed with trend or not
harmonic_order (int) – The harmonic order of the time-series regression
lambda (float) – Weight of previous observation in the monitoring process (memory). Valid range is [0,1], 1 corresponding to no memory and 0 to full memory
sensitivity (float) – Sensitivity parameter used in the computation of the monitoring boundaries. Lower values imply more sensitive monitoring
threshold_outlier (float) – Values bigger than threshold_outlier*sigma (extreme outliers) will get screened out during monitoring and will not contribute to updating the EWMA process value
**kwargs – Used to set internal attributes when initializing with .from_netcdf()

fit(dataarray, method='OLS', screen_outliers='Shewhart', L=5, **kwargs)

Stable history model fitting

The preferred fitting method for this monitoring type is 'OLS' with outlier screening 'Shewhart'. It requires a control limit parameter L. See nrt.outliers.shewart for more details

nrt.monitor.iqr module

class nrt.monitor.iqr.IQR(trend=True, harmonic_order=3, sensitivity=1.5, mask=None, boundary=3, **kwargs)

Bases: BaseNrt

Online monitoring of disturbances based on interquartile range

Reference:
https://stats.stackexchange.com/a/1153

mask

A 2D numpy array containing pixels that should be monitored (1) and not (0). The mask may be updated following history period stability check, and after a call to monitor following a confirmed break. Values are as follow. {0: 'Not monitored', 1: 'monitored', 2: 'Unstable history', 3: 'Confirmed break - no longer monitored'}

Type:: numpy.ndarray

trend

Indicate whether stable period fit is performed with trend or not

Type:: bool

harmonic_order

The harmonic order of the time-series regression

Type:: int

beta

3D array containing the model coefficients

Type:: np.ndarray

x

array of x coordinates

Type:: numpy.ndarray

y

array of y coordinates

Type:: numpy.ndarray

sensitivity

sensitivity of the monitoring. Lower numbers are high sensitivity. Value can’t be zero.

Type:: float

boundary

Number of consecutive observations identified as outliers to signal as disturbance

Type:: int

q25

25th percentile of residuals

Type:: numpy.ndarray

q75

75th percentile of residuals

Type:: numpy.ndarray

detection_date

2D array signalling detection date of disturbances in days since 1970-01-01

Type:: numpy.ndarray

Parameters:

mask (numpy.ndarray) – A 2D numpy array containing pixels that should be monitored marked as 1 and pixels that should be excluded (marked as 0). Typically a stable forest mask when doing forest disturbance monitoring. If no mask is supplied all pixels are considered and a mask is created following the fit() call
trend (bool) – Indicate whether stable period fit is performed with trend or not
harmonic_order (int) – The harmonic order of the time-series regression
sensitivity (float) – sensitivity of the monitoring. Lower numbers are high sensitivity. Value can’t be zero.
boundary (int) – Number of consecutive observations identified as outliers to signal as disturbance
**kwargs – Used to set internal attributes when initializing with .from_netcdf()

fit(dataarray, method='OLS', **kwargs)

nrt.monitor.mosum module

class nrt.monitor.mosum.MoSum(trend=True, harmonic_order=2, sensitivity=0.05, mask=None, h=0.25, **kwargs)

Bases: BaseNrt

Monitoring using moving sums (MOSUM) of residuals

Implementation following method as implemented in R package bFast.

mask

A 2D numpy array containing pixels that should be monitored (1) and not (0). The mask may be updated following history period stability check, and after a call to monitor following a confirmed break. Values are as follow. {0: 'Not monitored', 1: 'monitored', 2: 'Unstable history', 3: 'Confirmed break - no longer monitored'}

Type:: numpy.ndarray

trend

Indicate whether stable period fit is performed with trend or not

Type:: bool

harmonic_order

The harmonic order of the time-series regression

Type:: int

x

array of x coordinates

Type:: numpy.ndarray

y

array of y coordinates

Type:: numpy.ndarray

sensitivity

sensitivity of the monitoring. Lower numbers correspond to lower sensitivity. Equivalent to significance level ‘alpha’ with which the boundary is computed

Type:: float

boundary

process boundary for each time series. Calculated from alpha and length of time series.

Type:: numpy.ndarray

sigma

Standard deviation for normalized residuals in history period

Type:: numpy.ndarray

histsize

Number of non-nan observations in history period

Type:: numpy.ndarray

n

Total number of non-nan observations in time-series

Type:: numpy.ndarray

critval

Critical test value corresponding to the chosen sensitivity

Type:: float

h

Moving window size relative to length of the history period. Can be one of 0.25, 0.5 and 1

Type:: float

winsize

2D array with absolute window size. Computed as h*histsize

Type:: numpy.ndarray

window

3D array containing the current values in the window

Type:: numpy.ndarray

detection_date

2D array signalling detection date of disturbances in days since 1970-01-01

Type:: numpy.ndarray

Parameters:

mask (numpy.ndarray) – A 2D numpy array containing pixels that should be monitored marked as 1 and pixels that should be excluded (marked as 0). Typically a stable forest mask when doing forest disturbance monitoring. If no mask is supplied all pixels are considered and a mask is created following the fit() call
trend (bool) – Indicate whether stable period fit is performed with trend or not
harmonic_order (int) – The harmonic order of the time-series regression
sensitivity (float) – sensitivity of the monitoring. Lower numbers correspond to lower sensitivity. Equivalent to significance level ‘alpha’ with which the boundary is computed
h (float) – Moving window size relative to length of the history period. Can be one of 0.25, 0.5 and 1
**kwargs – Used to set internal attributes when initializing with .from_netcdf()

fit(dataarray, method='ROC', alpha=0.05, **kwargs)

Stable history model fitting

If method 'ROC' is used for fitting, the argument alpha has to be passed.

Parameters:

dataarray (xr.DataArray) – xarray Dataarray including the historic data to be fitted
method (string) – Regression to use. See _fit() for info.
alpha (float) – Significance level for 'ROC' stable fit.
**kwargs – to be passed to _fit

get_process()

property process

set_process(x)

Module contents

class nrt.monitor.BaseNrt(mask=None, trend=True, harmonic_order=3, save_fit_start=False, beta=None, x_coords=None, y_coords=None, process=None, boundary=None, detection_date=None, fit_start=None, update_mask=True, **kwargs)

Bases: object

Abstract class for Near Real Time change detection

Every new change monitoring approach should inherit from this abstract class and must implement the abstract methods fit(), monitor() and report(). It contains generic method to fit time-series harmonic-trend regression models, backward test for stability, dump the instance to a netcdf file and reload a dump.

mask

A 2D numpy array containing pixels that should be monitored (1) and not (0). The mask may be updated following historing period stability check, and after a call to monitor following a confirmed break. Values are as follow. ``{0: ‘Not monitored’,

1: ‘monitored’, 2: ‘Unstable history’, 3: ‘Confirmed break - no longer monitored’, 4: ‘Not enough observations - not monitored’}``

Type:: numpy.ndarray

trend

Indicate whether stable period fit is performed with trend or not

Type:: bool

harmonic_order

The harmonic order of the time-series regression

Type:: int

x

array of x coordinates

Type:: numpy.ndarray

y

array of y coordinates

Type:: numpy.ndarray

process

2D numpy array containing the process value for every pixel

Type:: numpy.ndarray

boundary

Process boundary for all pixels or every pixel individually

Type:: Union[numpy.ndarray, int, float]

detection_date

2D array signalling detection date of disturbances in days since 1970-01-01

Type:: numpy.ndarray

fit_start

2D integer array reporting start of history period in days since UNIX Epoch. Start of history period only varies when using stable fitting algorithms

Type:: numpy.ndarray

update_mask

Specifies whether to update the mask and halt further monitoring when values exceed boundary limits during a .monitor() call. A True value indicates that crossing the boundary limits will trigger a mask update and stop successive observation monitoring.

Type:: bool

Parameters:

mask (numpy.ndarray) – A 2D numpy array containing pixels that should be monitored marked as 1 and pixels that should be excluded (marked as 0). Typically a stable forest mask when doing forest disturbance monitoring. If no mask is supplied all pixels are considered and a mask is created following the fit() call
trend (bool) – Indicate whether stable period fit is performed with trend or not
harmonic_order (int) – The harmonic order of the time-series regression
save_fit_start (bool) – If start of the fit should be reported in the model. Only applicable to stable fits (e.g. ‘ROC’, ‘CCDC-stable’). If true, the data will be saved in the attribute fit_start
x_coords (numpy.ndarray) – x coordinates
y_coords (numpy.ndarray) – y coordinates
process (numpy.ndarray) – 2D numpy array containing the process value for every pixel
boundary (Union[numpy.ndarray, int, float]) – Process boundary for all pixels or every pixel individually
detection_date (numpy.ndarray) – 2D array signalling detection date of disturbances in days since 1970-01-01
fit_start (numpy.ndarray) – 2D integer array reporting start of history period in days since UNIX Epoch. Start of history period only varies when using stable fitting algorithms
update_mask (bool) – Specifies whether to update the mask and halt further monitoring when values exceed boundary limits during a .monitor() call. A True value (default) indicates that crossing the boundary limits will trigger a mask update and stop successive observation monitoring.

static build_design_matrix(dataarray, trend=True, harmonic_order=3)

Build a design matrix for temporal regression from xarray DataArray

Parameters:

trend (bool) – Whether to include a temporal trend or not
harmonic_order (int) – The harmonic order to use (1 corresponds to annual cycles, 2 to annual and biannual cycles, etc)

Returns:

A design matrix to be passed to be passed to e.g. the: _fit() method

Return type:

numpy.ndarray

abstract fit()

classmethod from_netcdf(filename, **kwargs)

monitor(array, date, update_mask=None)

Monitor given a new acquisition

The method takes care of (1) predicting the expected pixels values, (2) updating the process value, and (3) updating self.mask in case a break is confirmed

Parameters:

array (np.ndarray) – 2D array containing the new acquisition to be monitored
date (datetime.datetime) – Date of acquisition of data contained in the array
update_mask (bool) – Override update_mask instance attribute

predict(date)

Predict the expected values for a given date

Parameters:: date (datetime.datetime) – The date to predict
Returns:: A 2D array of predicted values
Return type:: numpy.ndarray

report(filename, layers=['mask', 'detection_date'], driver='GTiff', crs=CRS.from_epsg(3035), dtype=<class 'numpy.int16'>)

Write the result of reporting to a raster geospatial file

Parameters:

layers (list) –
A list of strings indicating the layers to include in the report. Valid options are 'mask' (the main output layer containing confirmed breaks, non-monitored pixels, etc), 'detection_date' (an integer value matching each confirmed break and indicating the date

the break was confirmed in days since epoch), 'process' (the process value). The process value has a different meaning and interpretation for each monitoring method.
dtype (type) – The datatype of the stacked layers. Note that when returning process value for MoSum, CuSum or EWMA the dtype should be set to a float type to retain values

set_xy(dataarray)

to_netcdf(filename)

property transform