What would be proper way to store data from file and validate data from file

Question

what is proper way to save all lines from text file to objects. I have .txt file something like this

0001Marcus Aurelius          20021122160   21311
0002William  Shakespeare     19940822332   11092
0003Albert Camus             20010715180   01232

From this file I know position of each data that is written in file, and all data are formatted.

Line number is from 0 to 3
Book author is from 4 to 30
Publish date is from 31 to 37
Page num. is from 38 to 43
Book code is from 44 to 49

I made class Data which holds information about start, end position, value, error.

Then I made class Line that holds list of type Data, and list that holds all error founded from some line. After load data from line to object Data I loop through lineError and add errors from all line to list, because I need to save errors from each line to database.

My question is this proper way to save data from file to object and after processing same data saving to database, advice for some better approach? Is better approach to create custum attribute which holds position of every record? For this I can't use third party library.

public class Data
{
    public int startPosition = 0;
    public int endPosition = 0;
    public object value = null;
    public string fieldName = "";
    public Error error = null;

    public Data(int start, int end, string name)
    {
        this.startPosition = start;
        this.endPosition = end;
        this.fieldName = name;
    }

    public void SetValueFromLine(string line)
    {
        string valueFromLine = line.Substring(this.startPosition, 
                               this.endPosition - this.startPosition);
        // if else statment that checks validity of data (lenght, empty 
         value) 
         this.value = valueFromLine;
     }

 }


 public class Line
 {
     public List<Data> lineData = new List<Data>();
     public List<Error> lineError = new List<Error>();

     public Line()
     {
         AddObjectDataToList();
     }

     public void AddObjectDataToList()
     {
         lineData.Add(new Data(0, 3, "lineNumber"));
         lineData.Add(new Data(4, 30, "bookAuthor"));
         lineData.Add(new Data(31, 37, "publishData"));
         lineData.Add(new Data(38, 43, "pageNumber"));
         lineData.Add(new Data(44, 49, "bookCode"));
     }

     public void LoadLineDataToObjects(string line)
     {
         foreach(Data s in lineData)
         {
             s.SetValueFromLine(line);
         }
      }

      public void GetAllErrorFromData()
      {
           foreach (Data s in lineData)
           {
               if(s.error != null)
               {
                    lineError.Add(s.error);
               }

           } 
      }

   } 

   public class File
   {
     public string fileName;
     public List<Line> lines = new List<Line>();
   }

Answer 1

from what I understood, you need a Model class, since you don't have one for the current data. I assume the file that holds the information is saved by other third-party application, and you're working on a middle-ware application in which would take that file and insert the data into the database.

If possible, try to find another approach than reading the file and try parse each line of it to convert them into an object model. If the other end can process CVS, a table view from the database if any, or an API that would retrieve the information you need, these would be a much safer approaches.

If there is no other way than reading the file and parsing each line. You can still do your idea, but hence, you must know that you need the replace both (the other end application, and this one) with an application that process the data better (combining both application processes into one).

The class you need :

// Book Model
public class Book
{
    // position start index: 0 
    //  Length: 4
    public int LineNumber { get; set; }

    // position start index: 4 
    //  Length: 25
    public string Author { get; set; }

    // position start index: 29
    //  Length: 8   
    public DateTime Publish { get; set; }

    // position start index: 37
    //  Length: 5
    public int PageNumber { get; set; }

    // position start index: 43
    //  Length: 5
    public string Code { get; set; }

    public Book(string line)
    {
        // normal validation
        if (string.IsNullOrEmpty(line)) { throw new ArgumentNullException(nameof(line)); }

        // business validation 
        if (line.Length != 48) { throw new InvalidOperationException(nameof(line)); }

        LineNumber = int.TryParse(ParseValue(line, 0, 4), out int lineResult) ? lineResult : -1;

        Author = ParseValue(line, 4, 25);

        Publish = DateTime.TryParse(ParseValue(line, 29, 8), out DateTime publishResult) ? publishResult : DateTime.MinValue;

        PageNumber = int.TryParse(ParseValue(line, 37, 5), out int pageResult) ? pageResult : -1;

        Code = ParseValue(line, 43, 5);
    }

    private string ParseValue(string line, int startIndex, int endIndex)
    {
        // get 
        var value = line.Substring(startIndex, endIndex).Trim();

        // check if is it a white space return null 
        // we need to discard whitespace initialization.
        // this could be a business requirment where all data should have real data 
        // if this is a requirment, then throw an exception instead of null
        if (value.Length == 0) { return null; }

        return value;
    }
}

Then, you stream the file and create a Book from each line or implement GetBooks method example :

public IEnumerable<Book> GetBooks(string filePath)
{
    if(string.IsNullOrEmpty(filePath)) { throw new ArgumentNullException(nameof(filePath)); }

    using (var fs = File.Open(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
    using (var bs = new BufferedStream(fs))
    using (var sr = new StreamReader(bs))
    {
        string line;
        while ((line = sr.ReadLine()) != null)
        {
            yield return new Book(line);
        }
    }
}

then you could do this :

var books = GetBooks(@"filepath");

foreach(var book in books)
{
    // do something with each book object. 
}

or you can use LINQ if you need as well.

UPDATE

Since you're looking for using Attribute you need to keep in mind, using attributes would put more work on your hands, as you would define them, and use Reflection to access its values, which would add more processing time to your code performance.

To create a new custom attribute you can do this :

/// <summary>
/// Implement StringPosition custom attribute
/// To define the start position and the length of the extracted string
/// </summary>
[AttributeUsage(AttributeTargets.Property)]
public class StringPositionAttribute : Attribute
{
    public int StartPosition { get; set; }

    public int Length { get; set; }

    public StringPositionAttribute(int startPosition, int endPosition)
    {
        StartPosition = startPosition;
        Length = endPosition;
    }
}

Now, you can use [StringPosition(startIndex, length)] attribute on the class properties.

Example :

// Book Model
public class Book
{
    [StringPosition(0, 4)]
    public int LineNumber { get; set; }

    [StringPosition(4, 25)]
    public string Author { get; set; }

    [StringPosition(29, 8)]
    public DateTime Publish { get; set; }

    [StringPosition(37, 5)]
    public int PageNumber { get; set; }

    [StringPosition(43, 5)]
    public string Code { get; set; }
}

now, by doing this, we can create a method where it loop over all of these properties and extract the targeted string based on the values of their attributes, then save the results for each.

private void ParseValues(string line)
{
    foreach(var property in this.GetType().GetProperties())
    {
        // Get property attribute
        var attribute = (StringPositionAttribute)property.GetCustomAttribute(typeof(StringPositionAttribute));

        // Get the start index value from the attribute
        var startIndex = attribute.StartPosition;

        // Get the length value from the attribute
        var length = attribute.Length;

        // parse the values to get the string
        var value = ParseValue(line, startIndex, length);

        // set the value of the property for this instance
        property.SetValue(this, Convert.ChangeType(value, property.PropertyType), null);
    }
}

now use this method in the constructor :

public Book(string line)
{
    // normal validation
    if (string.IsNullOrEmpty(line)) { throw new ArgumentNullException(nameof(line)); }

    // business validation 
    if (line.Length != 48) { throw new InvalidOperationException(nameof(line)); }

    // do the parsing
    ParseValues(line);
}

this is a quick examples to give you a boost start, the attribute can be useful in many cases, such as validation, or messages ..etc. But I believe your work does not need that much of work.

I hope this would give you the boost you need.