Data Utilities

The pygeostat.datautils module includes data utility functions for desurveying drillhole data, compositing, labeling, and other data manipulation tasks.

Desurveying

Drillhole Class

class pygeostat.datautils.Drillhole(holeid=None, collarx=None, collary=None, collarz=None)

This class contains specific drill hole data and metadata.

Drill hole classes may be created or generated using pygeostat functions. This is primarily used for desurveying drill hole data.

Parameters:

• holeid (str) – Hole Id

• collarx (numeric) – x coordinate

• collary (numeric) – y coordinate

• collarz (numeric) – z coordinate

delx(dist, inclination, bearing)

Calculate change in X

Parameters:

• self

• dist (float) – distance along direction

• inclination (float)

• bearing (float)

Returns:

A float value of the change in distance

Return type:

Value (float)

dely(dist, inclination, bearing)

Calculate change in y

Parameters:

• self

• dist (float) – distance along direction

• inclination (float)

• bearing (float)

Returns:

A float value of the change in distance

Return type:

Value (float)

delz(dist, inclination, bearing)

Calculate change in z

Parameters:

• self

• dist (float) – distance along direction

• inclination (float)

• bearing (float) – Not currently used in the calculation

Returns:

A float value of the change in distance

Return type:

Value (float)

getxyz(along, null)

Return X, Y, Z values given the length along the drill hole

Parameters:

• self

• along (numeric) – Distance along the drill hole

Returns:

x, y, z (float)

set_desurvey

pygeostat.datautils.set_desurvey(collarfl, surveyfl, along_name, azimuth_name, inclination_name)

Sets up for desurveying returning a dictionary of DrillHole objects

Parameters:

• collarfl (DataFile) – pygeostat DataFile with dh, x, y, z all set

• surveyfl (DataFile) – pygeostat DataFile with dh set

• along_name (str) – variable name of “along” in survey file

• azimuth_name (str) – variable name of “azimuth/bearing” in survey file

• inclination_name (str) – variable name of “inclination” in survey file

Returns:

Returns a dictionary of DrillHole objects

Return type:

drillholes (dict)

Code author: pygeostat development team - 2014-04-03

get_desurvey

pygeostat.datautils.get_desurvey(datafl, drillholes, inplace=True, x='X', y='Y', z='Z', null=None)

Gets the desurvey of a DataFile given froms and tos.

Parameters:

• datafl (DataFile) –

  pygeostat DataFile to be desurveyed with one of the options:

  1. just ifrom or just ito set to desurvey at exactly that distance

  2. both ifrom and ito set to desurvey halfway in between (ie: midpoint)

• drillholes (dict) – dictionary of Drillhole objects, obtained from set_desurvey

Keyword Arguments:

inplace (bool) – modifies the datafl.data to have X, Y and Z values otherwise returns a pandas dataframe with X, Y and Z

Compositing

set_comps

pygeostat.datautils.set_comps(datafl, complength)

Returns pandas dataframe with drillhole, compfrom, compto

Parameters:

• datafl (DataFile) – pygeostat DataFile with dh, ifrom and ito set

• complength (numeric) – length of composites

Returns:

pandas DataFrame with dh, ifrom and ito set for each composite

Return type:

comps (DataFrame)

get_comps

pygeostat.datautils.get_comps(comps, datafl, vartypes='continuous', null=None, nprocess=None)

Returns a pandas DataFrame with upscaled composites parallelizing across

drillholes.

Parameters:

• datafl (DataFile) – pygeostat DataFile with dh, ifrom, ito and at least 1 variable to upscale

• comps (DataFrame) – pandas DataFrame with datafl.dh, datafl.ifrom, datafl.ito

• locations (corresponding to the composite)

Keyword Arguments:

• vartypes (str OR dict) – ‘continuous’, ‘categorical’ or a dictionary of variables like:

• {'Cu' – ‘continuous’,’Facies’,’categorial’}

• null – the null value (values less than or equal to this will not be used)

Returns:

Pandas DataFrame with values of upscaled variable

Return type:

upscaled (DataFrame)

fast_comps

pygeostat.datautils.fast_comps(comps, datafl, null=None)

Returns a pandas DataFrame with upscaled composites and ASSUMES NO MISSING VALUES

Parameters:

• datafl (DataFile) – pygeostat DataFile with dh, ifrom, ito and at least 1 variable to upscale

• comps (DataFrame) – pandas DataFrame with datafl.dh, datafl.ifrom, datafl.ito

• locations (corresponding to the composite)

Keyword Arguments:

null – the null value (values less than or equal to this will not be used)

Returns:

Pandas DataFrame with value of upscaled variable

Return type:

upscaled (DataFrame)

Labeling

insert_real_idx

pygeostat.datautils.insert_real_idx(data, num_real=0, bindex=True, real_column='Realization', bi_column='BlockIndex')

This will insert realization index columns. By default it will use the griddef associated with the file.

Parameters:

• num_real (int) – If you do not have a griddef associated with the file you can tell it how many realizations there are

• bindex (bool) – True or False for adding a block index

• real_column (str) – Set the name of the column used for the Realizations Index

• bi_column (str) – Set the name of the column used for the Block Index

Process

If there are already a “real_column” or “bi_column” columns it will overwrite the values in these columns If the “real_column” and “bi_column” columns aren’t in the dataframe it will insert these columns at the front.

Code author: pygeostat development team 2015-09-30

make_labels

pygeostat.datautils.make_labels(prefix, num, padding=0)

Returns a series of lables combining a prefix and a number with leading zeros

Parameters:

• prefix (str) – any letter(s) that you want as the prefix (for example B for blockindex)

• num (int) – The number of labels you want.

• padding (int) – if given an integer value will pad the numbers with zeros until the prefix + the numbers equal the length of the padding value

Returns:

This will return a series with “n” number of labels starting from 1

Return type:

Series

Note

Barrowed from website http://pandas.pydata.org/pandas-docs/stable/advanced.html#advanced

Examples

Creating an array of labels

>>> label = gs.datautils.make_labels('R', 3, padding=3)
>>> label
>>> [R001, R002, R003]

Code author: pygeostat development team 2015-09-21

Utility Functions

round_sigfig

pygeostat.datautils.round_sigfig(value, sigfigs)

Round a float or integer to a specified number of significant figures. Also handles effectively zero, infinity, and negative infinity values.

From: http://stackoverflow.com/questions/3410976/

Parameters:

• value (int or float) – Value that requires rounding

• sigfigs (int) – Number of significant figures to round the value to

Returns:

Rounded value

Return type:

new_value (int or float)

Example

>>> gs.round_sigfig(-0.00032161, 3)
>>> -0.00322

Code author: pygeostat development team 2015-10-13

fileheader

pygeostat.datautils.fileheader(datafl, mute=False)

Read a GSLIB file from python and return the header information. Useful for large files.

Code author: pygeostat development team 2016-02-15

corrmatstr

pygeostat.datautils.corrmatstr(corrmat, fmt)

Converts a correlation matrix that is currently a numpy matrix or a pandas dataframe, into a space delimited string. Correlation matrix strings are required in the parameter files of CCG programs such as USGSIM and supersec.

Currently, this function is hard coded to return two formats as specified by the fmt argument, one for 'usgsim' and one for 'supersec'. 'usgsim' returns the full correlation matrix while 'supersec' returns only the upper triangle of the matrix, without the diagonal values.

Parameters:

• corrmat – Correlation matrix as either a pandas dataframe (pd.DataFrame) or numpy matrix (np.ndarray).

• fmt (str) – Indicate which format to return. Accepts only one of ['usgsim', 'supersec']

Returns:

Correlation matrix as a space delimited string.

Return type:

corrstr (str)

Code author: pygeostat development team 2016-03-15

slice_grid

pygeostat.datautils.slice_grid(data, griddef, orient, slice_number, slice_thickness=None, tmin=None)

Slice a 3-D grid.

Parameters:

• data – 1-D array or a tidy long-form dataframe with a single column containing the variable in question and each row is an observation

• griddef (GridDef) – A pygeostat GridDef class created using gs.GridDef

• orient (str) – Orientation to slice data. 'xy', 'xz', 'yz' are the only accepted values

• slice_number (int) – Grid cell location along the axis not plotted to take the slice of data to plot

• slice_thickness (int) – Number of slices around the selected slice_number to average the attributes

• tmin (float) – A minimum threshold value to filter/key out the data

Returns:

1-D array of the sliced data

Return type:

view (np.ndarray)

Code author: pygeostat development team 2014-04-19

slicescatter

pygeostat.datautils.slicescatter(data, orient, slice_number, slicetol, griddef=None, x=None, y=None, z=None)

Slice scattered data based on a GSLIB style grid definition.

Parameters:

• data (pd.DataFrame or gs.DataFile) – Dataframe where each column is a variable and each row is an observation. Must contain the coordinate columns required depending on the value of orient. If a gs.DataFile class is passed, its attribute griddef, x, y, and z will be extracted.

• var (str) – Column header of variable under investigation

• orient (str) – Orientation to slice data. 'xy', 'xz', 'yz' are the only accepted values

• slice_number (int) – Grid cell location along the axis not plotted to take the slice of data to plot

• slicetol (float) – Slice tolerance to plot point data (i.e. plot +/- slicetol from the center of the slice). Any negative value plots all data. Default is to plot all data.

• griddef (gs.GridDef) – A pygeostat GridDef class created using gs.GridDef. Required if the attribute cannot be retrieved from data if it is a gs.DataFile class.

• x (str) – Column header of x-coordinate. Required if the attribute cannot be retrieved from data if it is a gs.DataFile class.

• y (str) – Column header of x-coordinate. Required if the attribute cannot be retrieved from data if it is a gs.DataFile class.

• z (str) – Column header of x-coordinate. Required if the attribute cannot be retrieved from data if it is a gs.DataFile class.

Returns:

pd.DataFrame of the sliced data

Return type:

pointview (pd.DataFrane)

Code author: pygeostat development team - 2016-04-11

fixpath

pygeostat.datautils.fixpath(path)

Convert a file path to an absolute path if required and make sure there are only forward slashes.

If copying the path directly from windows explorer or something that will produce a path like that, make sure to indicate to python that the string is raw. This is done by placing a r in front of the string. For example:

>>> string = r"A string with backslashes \ \ \ \"

Example

Make sure to place an r in front of the string so funny things don’t happen. A simple call:

>>> gs.fixpath(r"D:\Data\data.dat")

Code author: pygeostat development team - 2016-02-07

is_numeric

pygeostat.datautils.is_numeric(s)

Returns true if a value can be converted to a floating point number

ensure_dir

pygeostat.datautils.ensure_dir(f)

Function to make sure that directory(s) exists and if not, create it

ensure_path

pygeostat.datautils.ensure_path(path)

Function ensures that all folders in a given path or list of paths are created if they do not exist

nearest_eucdist

pygeostat.datautils.nearest_eucdist(x, y=None, z=None)

Calculate the euclidean distance to the nearest sample for each sample.

Parameters:

x (np.array) – Array of the coordinate in the x direction

Keyword Arguments:

• y (np.array) – Array of the coordinate in the y direction

• z (np.array) – Array of the coordinate in the z direction

Returns:

Array of the euclidean distance to the nearest sample for each sample

Return type:

dist (np.array)

Code author: pygeostat development team - 2016-07-28

VTK Export

write_vtp

pygeostat.datautils.write_vtp(datafl, vartypes, drillholes, outflname, complength=5.0, null=None)

Generates a VTP file compatible with ParaView.

Parameters:

• datafl (DataFile) – pygeostat DataFile with dh, ifrom, ito and at least 1 variable to upscale

• vartypes (dict) – dictionary of variable types like {‘Grade’:’continuous’, ‘Category’:’categorical’}

• drillholes (dict) – dictionary of drillhole IDs to pygeostat DrillHole objects likely obtained by running: drillholes = gs.set_desurvey(collarfl, surveyfl, ‘Depth’, ‘Azimuth’, ‘Inclination’)

• outflname (str) – output VTP file to generate

Keyword Arguments:

• null – the null value (values equal to this will not be used)

• complength – this is the composite length which everything is regularized to which helps prevent visual artefacts and long triangles with a tube filter