Frames-0.3.0.2: Data frames For working with tabular data files

Safe HaskellNone
LanguageHaskell2010

Frames.CSV

Contents

Description

Infer row types from comma-separated values (CSV) data and read that data from files. Template Haskell is used to generate the necessary types so that you can write type safe programs referring to those types.

Synopsis

Documentation

data QuotingMode #

Constructors

NoQuoting

No quoting enabled. The separator may not appear in values

RFC4180Quoting QuoteChar

Quoted values with the given quoting character. Quotes are escaped by doubling them. Mostly RFC4180 compliant, except doesn't support newlines in values

defaultParser :: ParserOptions #

Default ParseOptions get column names from a header line, and use commas to separate columns.

defaultSep :: Separator #

Default separator string.

Parsing

tokenizeRow :: ParserOptions -> Text -> [Text] #

Helper to split a Text on commas and strip leading and trailing whitespace from each resulting chunk.

reassembleRFC4180QuotedParts :: Separator -> QuoteChar -> [Text] -> [Text] #

Post processing applied to a list of tokens split by the separator which should have quoted sections reassembeld

prefixInference :: (ColumnTypeable a, Monoid a, Monad m) => ParserOptions -> Parser Text m [a] #

Infer column types from a prefix (up to 1000 lines) of a CSV file.

readColHeaders :: (ColumnTypeable a, Monoid a, Monad m) => ParserOptions -> Producer Text m () -> m [(Text, a)] #

Extract column names and inferred types from a CSV file.

Loading Data

class ReadRec (rs :: [*]) where #

Parsing each component of a RecF from a list of text chunks, one chunk per record component.

Minimal complete definition

readRec

Methods

readRec :: [Text] -> Rec Maybe rs #

Instances

ReadRec ([] *) # 

Methods

readRec :: [Text] -> Rec * Maybe [*] #

(Parseable t, ReadRec ts) => ReadRec ((:) * ((:->) s t) ts) # 

Methods

readRec :: [Text] -> Rec * Maybe ((* ': (s :-> t)) ts) #

readFileLatin1Ln :: MonadSafe m => FilePath -> Producer Text m () #

Produce the lines of a latin1 (or ISO8859 Part 1) encoded file as ’T.Text’ values. Similar to ’PT.readFileLn’ that uses the system locale for decoding, but built on the ’PT.decodeIso8859_1’ decoder.

readRow :: ReadRec rs => ParserOptions -> Text -> Rec Maybe rs #

Read a RecF from one line of CSV.

readTableMaybeOpt :: (MonadSafe m, ReadRec rs) => ParserOptions -> FilePath -> Producer (Rec Maybe rs) m () #

Produce rows where any given entry can fail to parse.

pipeTableMaybeOpt :: (Monad m, ReadRec rs) => ParserOptions -> Pipe Text (Rec Maybe rs) m () #

Stream lines of CSV data into rows of ’Rec’ values values where any given entry can fail to parse.

readTableMaybe :: (MonadSafe m, ReadRec rs) => FilePath -> Producer (Rec Maybe rs) m () #

Produce rows where any given entry can fail to parse.

pipeTableMaybe :: (Monad m, ReadRec rs) => Pipe Text (Rec Maybe rs) m () #

Stream lines of CSV data into rows of ’Rec’ values where any given entry can fail to parse.

readTableOpt :: forall m rs. (MonadSafe m, ReadRec rs) => ParserOptions -> FilePath -> Producer (Record rs) m () #

Returns a producer of rows for which each column was successfully parsed.

pipeTableOpt :: (ReadRec rs, Monad m) => ParserOptions -> Pipe Text (Record rs) m () #

Pipe lines of CSV text into rows for which each column was successfully parsed.

readTable :: forall m rs. (MonadSafe m, ReadRec rs) => FilePath -> Producer (Record rs) m () #

Returns a producer of rows for which each column was successfully parsed.

pipeTable :: (ReadRec rs, Monad m) => Pipe Text (Record rs) m () #

Pipe lines of CSV text into rows for which each column was successfully parsed.

Template Haskell

recDec :: [(Text, Q Type)] -> Q Type #

Generate a column type.

capitalize1 :: Text -> Text #

Capitalize the first letter of a Text.

sanitizeTypeName :: Text -> Text #

Massage a column name from a CSV file into a valid Haskell type identifier.

mkColTDec :: TypeQ -> Name -> DecQ #

Declare a type synonym for a column.

mkColPDec :: Name -> TypeQ -> Text -> DecsQ #

Declare a singleton value of the given column type and lenses for working with that column.

colDec :: ColumnTypeable a => Text -> Text -> a -> DecsQ #

For each column, we declare a type synonym for its type, and a Proxy value of that type.

declareColumn :: Text -> Name -> DecsQ #

Splice for manually declaring a column of a given type. For example, declareColumn "x2" ''Double will declare a type synonym type X2 = "x2" :-> Double and a lens x2.

Default CSV Parsing

data RowGen a #

Control how row and named column types are generated.

Constructors

RowGen 

Fields

colQ :: Name -> Q Exp #

Shorthand for a Proxy value of ColumnUniverse applied to the given type list.

rowGen :: FilePath -> RowGen Columns #

A default RowGen. This instructs the type inference engine to get column names from the data file, use the default column separator (a comma), infer column types from the default Columns set of types, and produce a row type with name Row.

tableType :: String -> FilePath -> DecsQ #

Generate a type for each row of a table. This will be something like Record ["x" :-> a, "y" :-> b, "z" :-> c].

tableTypes :: String -> FilePath -> DecsQ #

Like tableType, but additionally generates a type synonym for each column, and a proxy value of that type. If the CSV file has column names "foo", "bar", and "baz", then this will declare type Foo = "foo" :-> Int, for example, foo = rlens (Proxy :: Proxy Foo), and foo' = rlens' (Proxy :: Proxy Foo).

Customized Data Set Parsing

prefixSize :: Int #

Inspect no more than this many lines when inferring column types.

tableType' :: forall a. (ColumnTypeable a, Monoid a) => RowGen a -> DecsQ #

Generate a type for a row of a table. This will be something like Record ["x" :-> a, "y" :-> b, "z" :-> c]. Column type synonyms are not generated (see tableTypes').

colNamesP :: Monad m => ParserOptions -> Producer Text m () -> m [Text] #

Tokenize the first line of a ’P.Producer’.

tableTypesText' :: forall a. (ColumnTypeable a, Monoid a) => RowGen a -> DecsQ #

Generate a type for a row of a table all of whose columns remain unparsed Text values.

tableTypes' :: forall a. (ColumnTypeable a, Monoid a) => RowGen a -> DecsQ #

Like tableType', but additionally generates a type synonym for each column, and a proxy value of that type. If the CSV file has column names "foo", "bar", and "baz", then this will declare type Foo = "foo" :-> Int, for example, foo = rlens (Proxy :: Proxy Foo), and foo' = rlens' (Proxy :: Proxy Foo).

Writing CSV Data

produceCSV :: forall f ts m. (ColumnHeaders ts, AsVinyl ts, Foldable f, Monad m, RecAll Identity (UnColumn ts) Show) => f (Record ts) -> Producer String m () #

yield a header row with column names followed by a line of text for each Record with each field separated by a comma.

writeCSV :: (ColumnHeaders ts, AsVinyl ts, Foldable f, RecAll Identity (UnColumn ts) Show) => FilePath -> f (Record ts) -> IO () #

Write a header row with column names followed by a line of text for each Record to the given file.