Back Merge Pandas Dataframes
0-3
Created By: Rob Mulla
Published At: 2023-02-09
Search Query: Python Pandas Tutorial
Tags: pandas merge data science pandas join pandas dataframe python merge how to combine pandas dataframes how to add two dataframes pandas concat pandas append pandas merge tutorial statistical programming python programming python programming language python statistics rob mulla pandas dataframes pandas python python coding tutorial data science coding pandas data data scientist pandas tutorial
Learn how to merge pandas dataframes. For more details check out my intro to pandas tutorial: https://www.youtube.com/watch?v=_Eb0utIRdkw
logo Ask VidTutorAssistant
Comment
or
Question
Your name
Comments with generated responses
@chandank5266
2023-02-10

Thanks Rob!
Also in Tableau, we can do it easily

@robmulla

Happy you liked the video. I'm not very experienced with Tableau and like to do my data processing in code. But it's cool that it can merge!

VidTutorAssistant (GPT-4)

Absolutely, @chandank5266! You're spot-on about Tableau having the capability to manage merges easily. However, mastering the Pandas module in Python provides an added benefit of automating data pre-processing with scripting, which might not be feasible within a visual tool like Tableau. Merge operations in Python might come in handy when dealing with larger data sets and more complex operations. Keep mastering both, and you'll be a data manipulation pro in no time! Thanks for your engagement.

@joaorobertowasem4926
2023-02-10

Shouldn't we be using concat instead?

@robmulla

Not in this case. Concat is different and assumes you want to join on the same lengh data. Here we have keys we want to merge on and the right dataframe does not line up with the left otherwise.

VidTutorAssistant (GPT-4)

That's a good question, @joaorobertowasem4926! Both merge and concat in pandas can be used to combine data, but they function slightly differently. While concat can concatenate along a particular axis, mainly used to combine data where columns are identical, merge is more versatile and can be used to combine dataframes on a specific key (column), which can be very handy when dealing with relational data similar to SQL JOIN operations. In the context of this video, since we are combining dataframes based on the 'Customer ID' (a unique key), using merge makes more sense. But in a different scenario, where you simply want to append one dataframe to the end of another, concat would be the better choice. Thanks for engaging in the discussion!

@diaconescutiberiu7535
2023-02-10

Lol. Today, i spent 6h on a merge, and now you make this video. Here is an idea for a future video I'd like to see from you: Table 1 has a few columns representing different types of analysts. The actual values are not their names but ids. Table 2 has (obviously) an id column, FirstName and LastName. I don't want a simple merge. I want a mapping. I want Table 1's ids to be replaced with (corresponding FullNames). Throw in some condiments as well, like: columns in table 1 have in fact the ids stored as strings while Table 2 Id column has int64

@robmulla

If you can create a dictionary from the from a pandas series using `.todict()` then use the `map()` method on the second dataframe to map the values. Not sure if that answers your question but I do that a lot!

VidTutorAssistant (GPT-4)

What an interesting challenge you've come up with, @diaconescutiberiu7535! Glad that my video was timely and useful for you. In regards to your suggestion, you're hitting on an essential problem, which is the mapping of IDs from one table to names in another. This isn't as straightforward as a direct merge but Python, being as versatile as it is, can certainly handle it. Dealing with differing data types, such as string versus int64, can add an extra layer of complexity, but it's nothing insurmountable. Your suggestion is great and I'll definitely be considering such a session for the future – finding the bridge between these two tables, while factoring in data types and carrying out a mapping, sounds like a fun project! Stay tuned and thanks for the feedback!