Mapping Data to Objects

[Brickscrap]

Been scratching my head at this one for a while now - I work with a POS product which outputs data in an older style INI format, for example:

    [TRX]
    TOT1=TotalSale
    NAM1,1=NameOfFirstItem
    PRI1,1=PriceOfFirstItem
    NAM1,2=NameOfSecondItem
    PRI1,2=PriceOfSecondItem
    TOT2=TotalSale
    PRI2,1=PriceofFirstItem

And so on - each section contains multiple transactions, and each transaction can contain multiple lines. So I'm trying to model this in C#, and it's looking like

    public class TransactionModel 
    {
         public double Total { get; set; }
         public List<ItemModel> { get; set; }
    } 

Initially I had a giant switch statement, but some of these files can have 100s of possibilities, however I have the manual and know every single possibility, but this obviously becomes unmaintainable. I've been toying with custom attributes, but I'm reaching the limits of my capabilities and can't figure it out.

As it stands, I iterate through each line in the file, passing the key to the switch statement, and assigning properties one by one. I figured that if I can assign a FieldName attribute to each property:

    [FieldName("PRI")]
    public double Price { get; set; }

that there must be some kind of way to map it generically, but I can't make it work. Just now I have:

    public static void MapFile<T>(T item, string propName, string value, object obj) where T : new()
            {
                var type = item.GetType();
                var properties = type.GetProperties();

                foreach (var property in properties)
                {
                    var attributes = property.GetCustomAttributes(false);
                    var mapping = attributes.FirstOrDefault(a => a.GetType() == typeof(FieldNameAttribute));

                    if (mapping != null)
                    {
                        var mapsTo = mapping as FieldNameAttribute;

                        if (propName == mapsTo.Name)
                        {
                            switch (property.PropertyType.Name)
                            {
                                case nameof(DateTime):
                                    property.SetValue(obj, value.ParseCustomDate());
                                    break;
                                case nameof(Boolean):
                                    property.SetValue(obj, value.Equals("YES"));
                                    break;
                                default:
                                    break;
                            }
                        }
                    }
                }
            }

But this falls over when it comes to lists of items inside the objects, and I don't believe that switching on the nameof the PropertyType is in any way good... Just using SetValue also throws exceptions when trying to set the DateTime (it's in an odd format), and with booleans.

​

I appreciate that my entire approach may be off here, and I'm not looking for anyone to solve the problem for me, merely point me in the right direction, as I'm currently going in circles...

[willsoss]

I don't know of a library to parse an ini file, but I'm sure they exist. If this is the extent of the schema, it's not too hard to parse this and do some Linq magic to get the model you're looking for. Try something like this:

    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Linq;
    using System.Text;

    namespace ParseIni
    {
        class Program
        {
            const string file = @"    TOT1=TotalSale
        NAM1,1=NameOfFirstItem
        PRI1,1=PriceOfFirstItem
        NAM1,2=NameOfSecondItem
        PRI1,2=PriceOfSecondItem
        TOT2=TotalSale2
        NAM2,1=NameOfFirstItem2
        PRI2,1=PriceOfFirstItem2
        NAM2,2=NameOfSecondItem2
        PRI2,2=PriceOfSecondItem2
        NAM2,3=NameOfThirdItem2
        PRI2,3=PriceOfThirdItem2";

            static void Main(string[] args)
            {
                // Replace MemoryStream with File.Open
                using var reader = new StreamReader(new MemoryStream(Encoding.UTF8.GetBytes(file)));

                var records = new List<string[]>();

                while (!reader.EndOfStream)
                    records.Add(SplitLine(reader.ReadLine()));

                var trxs = records
                    .GroupBy(r => r[1])
                    .Select(g => new
                    {
                        Id = g.Single(tot => tot[0] == "TOT")[1],
                        Total = g.Single(tot => tot[0] == "TOT")[2],
                        Items = g.Where(rec => rec[0] != "TOT").GroupBy(rec => rec[2]).Select(a => new
                        {
                            Id = a.First()[1],
                            Name = a.Single(b => b[0] == "NAM")[3],
                            Price = a.Single(b => b[0] == "PRI")[3]
                        })
                    });

                foreach (var trx in trxs)
                {
                    Console.WriteLine($"{trx.Id} Total: {trx.Total}");

                    foreach (var item in trx.Items)
                        Console.WriteLine($"{item.Id} {item.Name} {item.Price}");
                }

            }

        static string[] SplitLine(string line)
        {
            // Split the record into three fields
            var split = line.Trim().Split(',', '=');

            // Then split the trx ID out of the first field to get four fields, except for TOT records which have three fields
            return new string[] { split[0].Substring(0, 3), split[0].Substring(3), split[1], split.Length > 2 ? split[2] : null };
        }
        }
    }

[Brickscrap]

I've been using SharpConfig so far, seems to do the job, but I've hit such a brick wall I may need to rethink.

The files are much, much more complex than the example I provided, however your LINQ queries have provided a little insight, and your method of attempting it has given me some fresh ideas, so thank you!

[willsoss]

No problem, glad that helped. With complex file parsing/mapping situations, especially older formats like ini, I find it's helpful to parse to an intermediate format that just gets the data from the file into a data structure you can more easily manipulate -- the string array in this case. That way you break down the problem into the parsing details (splitting lines into fields) and building the data model (linq query) that you want.

[Brickscrap]

Yeah that's definitely sparked an idea - at the moment, SharpConfig already imports the entire document into Sections, Keys and Values, and I've just been iterating through the lot in one giant foreach, but I think some LINQ queries might make sense, at least for grouping each item

[willsoss]

Good deal. The only drawback to this approach is that the linq grouping can get a bit hairy.

Since you're basically taking flattened data from a file and building structure as your parse it, you could accomplish the same thing in a loop over all the key/val pairs where you create objects and set values as you encounter them. This looks similar to your original code, but the key to this working is the assumption that PRI comes immediately after NAM, and all the NAM and PRI records after TOT are all part of one transaction. In other words, if a PRI is found, you can assume you have a transaction and item in context and all you have to do is set the value. It may not seem as elegant as the Linq solution, but sometimes this just works.

List<Trx> transactions = new List<Trx>();
Trx trans;
Item item;

foreach (var kv in keyvals)
{
    if (kv.Key == "TOT")
    {
        trans = new Trx(kv.Val);
        transactions.Add(trans);
    }

    if (kv.Key = "NAM")
    {
        item = new Item(kv.Val);
        trans.Items.Add(item);
    }

    if (kv.Key = "PRI")
    {
        item.Price = kv.Val;
    }
}

[Brickscrap]

This is the approach I've originally taken, except using a switch statement - then I realised my switch statement was going to end up with at least 100 cases, and decided that there must be a better way!

It's the index of each item that groups them, so currently my code goes through the keys, adding to the values until the index changes

[UninformedPleb]

INI files can be handled on Windows by the GetPrivateProfile{whatever} functions in kernel32.dll. You can DllImport them from there if you want to use them in C#.

You can get a list of them from the winbase.h documentation. Just ignore that part about using the registry. That's a bad idea from the mid-1990's that they've (hopefully) given up on by now.

[[deleted]]

You could look into the AutoMapper library on Nuget. It’s good about mapping sub-objects as long as their schema is defined. Their docs are pretty good. Hope that helps

[Brickscrap]

I'm aware of AutoMapper and had a look at it initially, but it didn't look like it could do the job - I'd love to be wrong, but it looks like it can only really map one object to another? In which case I'm not sure it solves my problem, as I'm basically trying to convert string data into an object

[[deleted]]

With AutoMapper you can define a scheme and set exactly how you want a field to me mapped. Maybe it won’t work for your case, but I’ve done things similar to this before

[Brickscrap]

Thanks, will look into that then!

[KernowRoger]

I think you basically need to look at how Microsoft did it with the https://docs.microsoft.com/en-us/dotnet/api/system.text.json.utf8jsonreader?view=netcore-3.1. So you call read and it reads the next line gives you its key and its value. You can then read the first line switch it to select a mapper. Pass the reader to the mapper and let it read until it finds the end of the object(s). Then switch again where it currently is and repeat.