8. Dynamic Data Transformation
By Bernd Klein. Last modified: 24 Mar 2024.
Definition
Dynamic data transformation refers to the process of altering or converting data in real-time or on-the-fly based on specific conditions, rules, or requirements. Unlike static data transformation, which applies fixed transformations to data regardless of context, dynamic data transformation adapts its transformations dynamically according to the data's characteristics or external factors.
This approach allows for flexibility and responsiveness in handling diverse datasets or changing requirements. Dynamic data transformation typically involves techniques such as conditional logic, parameterization, and automation to efficiently modify data as it flows through a system or process. It is commonly used in various domains including data integration, ETL (Extract, Transform, Load), data cleansing, and data analysis to ensure that data is appropriately formatted, structured, and enriched for downstream applications or analysis.
Live Python training
Enjoying this page? We offer live Python training courses covering the content of this site.
Product Class Example
Our example showcases a product class that could be utilized within a company or shop setting, e.g. a cheese shop.1
In the context of the provided Product class, when you change the currency using the set_currency method with the adapt_data parameter set to True, it dynamically adjusts the displayed prices and shipping costs based on the new currency without altering the original saved values. This process ensures that the user sees the values in the desired currency without permanently changing the underlying data. So this means that the process of dynamically changing the data also means that we are often seeing different values than are saved.
It is also more efficient to keep the data instead of permanently changing it. We just adapt the data "on demand":
class Product:
"""
A class representing a product with price and shipping cost.
Attributes:
conversion_rates (dict): A dictionary containing conversion rates from different currencies to USD.
"""
conversion_rates = {'USD': 1, 'EUR': 0.92, 'CHF': 0.90, 'GBP': 0.79}
def __init__(self, name, price, shipping_cost, currency='USD'):
"""
Initializes a Product object with the given parameters.
Args:
name (str): The name of the product.
price (float): The price of the product in the specified currency.
shipping_cost (float): The shipping cost of the product in the specified currency.
currency (str, optional): The currency code for price and shipping cost. Defaults to 'USD'.
"""
self.name = name
self._price = price
self._shipping_cost = shipping_cost
self.currency = currency
self._used_currency = currency
def set_currency(self, new_currency, adapt_data=False):
"""
Sets a new currency for the product and optionally adapts existing data.
Args:
new_currency (str): The new currency code.
adapt_data (bool, optional): Whether to adapt existing data to the new currency. Defaults to False.
"""
if self.currency != new_currency:
self.currency = new_currency
if adapt_data:
self._price = self.price
self._shipping_cost = self.shipping_cost
self._used_currency = new_currency
@property
def price(self):
"""
Property representing the price of the product in the specified currency.
Returns:
float: The price of the product.
"""
return self._convert_currency(self._price)
@property
def shipping_cost(self):
"""
Property representing the shipping cost of the product in the specified currency.
Returns:
float: The shipping cost of the product.
"""
return self._convert_currency(self._shipping_cost)
def _convert_currency(self, amount):
"""
Converts an amount from the internal currency to the specified currency.
Args:
amount (float): The amount to be converted.
Returns:
float: The converted amount.
"""
factor = Product.conversion_rates[self.currency] / Product.conversion_rates[self._used_currency]
return round(amount * factor, 2)
def __str__(self):
"""
Returns a string representation of the Product object.
Returns:
str: A string containing product details.
"""
return f"Product: {self.name}, Price: {self.price} {self.currency}, Shipping Cost: {self.shipping_cost} {self.currency}"
def show_saved_data(self):
"""
Prints the saved data of the Product object.
"""
outstr = f"Saved Data: {self.name=}, {self.currency=}, {self._used_currency=} {self._price=}, {self._shipping_cost=}"
print(outstr)
The mechanism maintains two currency attributes: currency denotes the current currency, i.e. the currency in which the user wants to see the data, while __used_currency represents the currency in which data was last adapted, i.e. in which the data ist still saved.
Whenever we call set_currency with a new currence new_currency and adapt_data is False, The data self._prize and self._shipping_costs will not be changed. We only set self.currency to new_currency.
But if we call set_currency with adapt_data=True we will actually change the data to the new currency by executing this code:
self._price = self.price
self._shipping_cost = self.shipping_cost
self._used_currency = new_currency
By using self._used_currency in conjunction with self.currency, the Product class can determine whether data needs to be converted when self.price or self.shipping_cost is accessed. This combination allows for dynamic and accurate currency conversion only when necessary, ensuring that data remains consistent and correctly represented across different currencies.
Let's instantiate a product to illustrate this functionality. Additionally, it's worth noting that in a real-world scenario, the show_saved_data method wouldn't typically be included in the class. It's primarily intended for debugging or educational purposes.
product = Product(name='Phone', price=500, shipping_cost=10, currency='USD')
print(product)
product.show_saved_data()
Product: Phone, Price: 500.0 USD, Shipping Cost: 10.0 USD Saved Data: self.name='Phone', self.currency='USD', self._used_currency='USD' self._price=500, self._shipping_cost=10
We can see that both self.currency and self._used_currency are set to USD, Additionally, we can also notice that self._price and self._shipping_cost are set to what we initialized them to.
We will set the currency to 'GBP':
product.set_currency('GBP')
print(product)
product.show_saved_data()
Product: Phone, Price: 395.0 GBP, Shipping Cost: 7.9 GBP Saved Data: self.name='Phone', self.currency='GBP', self._used_currency='USD' self._price=500, self._shipping_cost=10
We can notice that all the values execpt the attribute currency are the same as before. self.currency is set to 'GBP', whereas self._used_currency shows still the original currency 'USD'. The values for self._price and self._shipping_cost are not changed.
In the following, we change the currency to 'Eur', but this time we set adapt_data to True:
product.set_currency('EUR', adapt_data=True)
print(product)
product.show_saved_data()
Product: Phone, Price: 460.0 EUR, Shipping Cost: 9.2 EUR Saved Data: self.name='Phone', self.currency='EUR', self._used_currency='EUR' self._price=460.0, self._shipping_cost=9.2
Upon examining the preceding output, it becomes apparent that both self.currency and self._used_currency have been updated to EUR. Notably, the data has also been modified accordingly, with self._price and self._shipping_cost transformed from their original Dollar values to their corresponding Euro values.
We're now introducing a new class called Products, which employs our existing Product class. This serves also as a good example of class composition, also known as class aggregation.
The purpose of the Products class is to manage a collection of product instances. It provides functionality to add products to the collection and view the products in various formats or currencies. Additionally, it allows for displaying products' saved data. Overall, the Products class facilitates the organization and manipulation of multiple product instances collectively.
class Products:
def __init__(self):
self.product_list = []
def add_product(self, product):
self.product_list.append(product)
def view_products(self, currency='USD', adapt_data=False):
total_products = len(self.product_list)
print(f"Total Products: {total_products}")
print("Products:")
for product in self.product_list:
product.set_currency(currency, adapt_data)
print(product)
def view_products(self, currency='USD', adapt_data=False):
total_products = len(self.product_list)
print(f"Total Products: {total_products}")
print("Products:")
print(f"{'Product name':39s} Price Shipping")
for product in self.product_list:
product.set_currency(currency, adapt_data)
print(f"{product.name:35s} {product.price:7.2f} {product.shipping_cost:6.2f}")
def view_products_as_saved_data(self):
total_products = len(self.product_list)
print(f"Total Products: {total_products}")
print("Products:")
print(f"{'Product name':38s} Price Shipping")
for product in self.product_list:
product.show_saved_data()
In order to see our class in action, we require products. However, before proceeding, we need product names. We'll generate prices randomly and select a product name at random for each product.
import random
fantasy_product_names = [
"Elixir of Eternal Youth",
"Dragonfire Sword",
"Phoenix Feather Wand",
"Mermaid's Tears Necklace",
"Elven Cloak of Invisibility",
"Potion of Flying",
"Amulet of Wisdom",
"Crystal Ball of Fortune",
"Enchanted Mirror",
"Unicorn Horn Ring"
]
currencies = list(Product.conversion_rates.keys())
# Example usage:
products = Products()
# Adding products with different currencies and fantasy names
for i, name in enumerate(fantasy_product_names, start=1):
price = random.uniform(70, 1000)
products.add_product(Product(name=name, price=price, shipping_cost=price*0.03, currency=random.choice(currencies)))
currency = 'USD'
print(f"Viewing products in {currency}:")
products.view_products(currency=currency)
Viewing products in USD: Total Products: 10 Products: Product name Price Shipping Elixir of Eternal Youth 371.85 11.16 Dragonfire Sword 744.64 22.34 Phoenix Feather Wand 828.00 24.84 Mermaid's Tears Necklace 667.11 20.01 Elven Cloak of Invisibility 100.45 3.01 Potion of Flying 522.46 15.67 Amulet of Wisdom 253.32 7.60 Crystal Ball of Fortune 874.77 26.24 Enchanted Mirror 113.52 3.41 Unicorn Horn Ring 1159.80 34.79
By using the following code, we can view all the products in Swiss Francs, regardless of their internal currency representation. It's important to note that the values will not be internally converted to Swiss Francs if they are stored in a different currency.
currency = 'CHF'
print(f"Viewing products in {currency}:")
products.view_products(currency=currency)
Viewing products in CHF: Total Products: 10 Products: Product name Price Shipping Elixir of Eternal Youth 535.05 16.05 Dragonfire Sword 534.21 16.03 Phoenix Feather Wand 452.75 13.58 Mermaid's Tears Necklace 771.28 23.14 Elven Cloak of Invisibility 382.03 11.46 Potion of Flying 96.99 2.91 Amulet of Wisdom 287.74 8.63 Crystal Ball of Fortune 1048.68 31.46 Enchanted Mirror 555.72 16.67 Unicorn Horn Ring 320.77 9.62
The following output shows the internal representation:
print(f"Viewing products in {currency}:")
products.view_products_as_saved_data()
Viewing products in USD: Total Products: 10 Products: Product name Price Shipping Saved Data: self.name='Elixir of Eternal Youth', self.currency='USD', self._used_currency='EUR' self._price=342.1039188768496, self._shipping_cost=10.263117566305487 Saved Data: self.name='Dragonfire Sword', self.currency='USD', self._used_currency='GBP' self._price=588.2634995094336, self._shipping_cost=17.647904985283006 Saved Data: self.name='Phoenix Feather Wand', self.currency='USD', self._used_currency='USD' self._price=828.0039691049606, self._shipping_cost=24.840119073148816 Saved Data: self.name="Mermaid's Tears Necklace", self.currency='USD', self._used_currency='USD' self._price=667.1149208358245, self._shipping_cost=20.013447625074733 Saved Data: self.name='Elven Cloak of Invisibility', self.currency='USD', self._used_currency='CHF' self._price=90.40778429170587, self._shipping_cost=2.712233528751176 Saved Data: self.name='Potion of Flying', self.currency='USD', self._used_currency='GBP' self._price=412.74000821657603, self._shipping_cost=12.38220024649728 Saved Data: self.name='Amulet of Wisdom', self.currency='USD', self._used_currency='GBP' self._price=200.11904978189625, self._shipping_cost=6.003571493456887 Saved Data: self.name='Crystal Ball of Fortune', self.currency='USD', self._used_currency='CHF' self._price=787.296039246531, self._shipping_cost=23.618881177395927 Saved Data: self.name='Enchanted Mirror', self.currency='USD', self._used_currency='GBP' self._price=89.68419860808669, self._shipping_cost=2.6905259582426004 Saved Data: self.name='Unicorn Horn Ring', self.currency='USD', self._used_currency='GBP' self._price=916.2441702805728, self._shipping_cost=27.487325108417185
The following code gives us another view at the data in Swiss Francs, but this time all the data will be internally changed also. This is due to the fact that we set adapt_data to True:
currency = 'CHF'
print(f"Viewing products in {currency}:")
products.view_products(currency=currency, adapt_data=True)
Viewing products in CHF: Total Products: 10 Products: Product name Price Shipping Elixir of Eternal Youth 334.67 10.04 Dragonfire Sword 670.17 20.11 Phoenix Feather Wand 745.20 22.36 Mermaid's Tears Necklace 600.40 18.01 Elven Cloak of Invisibility 90.41 2.71 Potion of Flying 470.21 14.11 Amulet of Wisdom 227.98 6.84 Crystal Ball of Fortune 787.30 23.62 Enchanted Mirror 102.17 3.07 Unicorn Horn Ring 1043.82 31.31
print(f"Viewing products in {currency}:")
products.view_products_as_saved_data()
Viewing products in CHF: Total Products: 10 Products: Product name Price Shipping Saved Data: self.name='Elixir of Eternal Youth', self.currency='CHF', self._used_currency='CHF' self._price=334.67, self._shipping_cost=10.04 Saved Data: self.name='Dragonfire Sword', self.currency='CHF', self._used_currency='CHF' self._price=670.17, self._shipping_cost=20.11 Saved Data: self.name='Phoenix Feather Wand', self.currency='CHF', self._used_currency='CHF' self._price=745.2, self._shipping_cost=22.36 Saved Data: self.name="Mermaid's Tears Necklace", self.currency='CHF', self._used_currency='CHF' self._price=600.4, self._shipping_cost=18.01 Saved Data: self.name='Elven Cloak of Invisibility', self.currency='CHF', self._used_currency='CHF' self._price=90.41, self._shipping_cost=2.71 Saved Data: self.name='Potion of Flying', self.currency='CHF', self._used_currency='CHF' self._price=470.21, self._shipping_cost=14.11 Saved Data: self.name='Amulet of Wisdom', self.currency='CHF', self._used_currency='CHF' self._price=227.98, self._shipping_cost=6.84 Saved Data: self.name='Crystal Ball of Fortune', self.currency='CHF', self._used_currency='CHF' self._price=787.3, self._shipping_cost=23.62 Saved Data: self.name='Enchanted Mirror', self.currency='CHF', self._used_currency='CHF' self._price=102.17, self._shipping_cost=3.07 Saved Data: self.name='Unicorn Horn Ring', self.currency='CHF', self._used_currency='CHF' self._price=1043.82, self._shipping_cost=31.31
Footnotes
1 Sorry, we couldn't resist the temptation of adding a touch of humor with a reference to the iconic "Cheese Shop" from Monty Python's Flying Circus sketch. It's a classic that often finds its way into discussions about Python programming, making appearances in many reputable tutorials and books on the subject.
Live Python training
Enjoying this page? We offer live Python training courses covering the content of this site.
Upcoming online Courses