Posts

Showing posts with the label models

Featured Post

8 Ways to Optimize AWS Glue Jobs in a Nutshell

-
Image
  Improving the performance of AWS Glue jobs involves several strategies that target different aspects of the ETL (Extract, Transform, Load) process. Here are some key practices. 1. Optimize Job Scripts Partitioning : Ensure your data is properly partitioned. Partitioning divides your data into manageable chunks, allowing parallel processing and reducing the amount of data scanned. Filtering : Apply pushdown predicates to filter data early in the ETL process, reducing the amount of data processed downstream. Compression : Use compressed file formats (e.g., Parquet, ORC) for your data sources and sinks. These formats not only reduce storage costs but also improve I/O performance. Optimize Transformations : Minimize the number of transformations and actions in your script. Combine transformations where possible and use DataFrame APIs which are optimized for performance. 2. Use Appropriate Data Formats Parquet and ORC : These columnar formats are efficient for storage and querying, signif
Post a Comment
Read more

2 Top Python Libraries to Create ML model

-
Image
To Create a Model of Machine Learning in Python, you need TWO libraries. One is 'NUMPY' and the other one is 'PANDAS'. 2 Top Libraries You Need To Build a model of Machine learning you need the right kind of data. So, use the data for your project should be refined. Else, it will not produce correct results. The prime steps are Data Analysis and Data Preprocessing. NUMPY - It has the capabilities of Calculations. PANDAS- It has the capabilities of Data processing. How Install Python Machine Learning Libraries  import  NumPy as np # linear algebra import pandas as PD # data processing, CSV file I/O (e.g. PD.read_csv) How to Check NumPy/Pandas installed After '.' you need to give double underscore on both sides of the version.  How Many Types of Data You Need You need two types of data. One is data to build a model and the other one is data you need to test the model. Raw data Evaluate-data How to Build a Model Flowchart I have given a f
1 comment
Read more

Real thoughts on IBM power8 servers to use on analytics

-
Image
IBM Servers International Business Machines Corp, in its latest attempt at reviving demand for its hardware products, is launching high-end system servers that it says are 50 times faster than its closest competitor at analysing data.  The POWER8 servers , the product of a $2.4 billion, three-year investment, are part of the company's decade-long shift to higher-value hardware technology.    IBM  said the machines are 50 times faster than the low-end x86-based servers it sold to Chinese PC maker  Lenovo  Group Ltd in January.  The technology services provider said on Wednesday it hopes the servers, designed for large-scale computing, will appeal to clients looking to manage new types of social and mobile computing and mass amounts of data. Last week, the company reported its lowest quarterly revenue in five years, weighed down by falling demand for its storage and server products. IBM dominates the higher-end server market with 57 percent market share, according to res
Post a Comment
Read more