In the previous installment of this miniseries, we introduced the concept of the Part Description string, which is a single string containing multiple attribute values without specifying attribute names. We also discussed briefly why and how this could cause problems for engineers working on part designs. Today, we would like to show you how we solved the issue of handling these values.
To refresh your memory, our customer asked us to provide the ability to create structured data out of unstructured Part Description values. The picture below shows the idea:
Thus, Rules Engine was brought to life as part of Encompass Data Loader Framework. Our first implementation worked pretty well, and the customer was happy. Unfortunately, it was difficult to maintain for us and adding new functionality was unnecessarily complicated. We learned a lot from that experience. And, after some time passed, we took lessons learned and pain points from the previous version of Rules Engine and redesigned it from the ground up.
The new Rules Engine has four major characteristics:
- Customers must be able to express their formal unstructured data specifications via XML.
- In rare cases, where the XML definition cannot represent the logic required to extract structured data from the unstructured input, support for the user-supplied scripts is available.
- Ability to perform data cleanup and customized transformations if needed
- Ability to detect data issues (like missing data, invalid data, invalid order of data, etc…)
Attribute value extraction is the primary goal of the Rules Engine. You have unstructured data on the input, and you want to see structured data on the output. You also want to know the names of the attributes based on the values in the unstructured data. If your company data policy for a given part type says that some attributes must be always present, you’ll want to know if they’re missing. If the same policy defines an attribute as optional, we can tell you whether or not it’s present.
If policy is strict on the order of the attribute values, the Rules Engine can verify that against each instance of unstructured data and show you where the order differs from what you would expect. We also let you define which values are allowed for each attribute. If we detect a value, which is not allowed, the Rules Engine can alert you. If there is some garbage data, which should not be there in the first place – yes, we can detect that, as well. We even support replacing predefined attribute values with their ”better” equivalents, should you have such a need.
The best part is: you can control almost all of that by simply changing the extraction rule definitions in XML. Seriously. No coding required. In those rare cases where you have a specification, which is so complex that XML configuration does not suffice, scripting support is there to help.
What You See Is What You Fix
It’s well known fact that users feedback is one of the best ways to improve the quality of your product. In manufacturing companies that use tools like Encompass on daily basis, there might be thousands of active users. These users are frequently engineers who can spot data issues when they see them, which is why Encompass Product Navigation has a nice module called Data Admin. Data Admin provides a visual presentation of how the Encompass Rules Engine extracts attributes from the specific Part Description values, in the form of color-coded markups.
Thanks to that, users can catch actual data issues or cases where the Rules Engine extraction rules configuration does not match the actual company specification. Users can export a list of their findings and send it to whoever is responsible for improving Data Quality in their organization.
And Much More
We hope you enjoyed a brief tour of one of the ways Perception makes consolidating and organizing product data easier. The Rules Engine is just one small piece of our Encompass Data Loader Framework. We can do virtually anything with your data in order to make it more useful for you when working with our Encompass navigation applications. Meaningful and searchable data is what we specialize in here at Perception, and we’re constantly improving our Encompass Data Loader Framework to deliver that to your organization faster and easier.
If you’re interested in finding out more, don’t hesitate to contact us. We’ll be happy to help!