DAWD 02-5-1 - Demo - Delta Commands in Databricks SQL
37 |
The two fields below are used to customize queries used in this course. Enter your schema (database) name and username, and press "Enter" to populate necessary information in the queries on this page.
38 |
39 |
40 |
Schema Name:
41 |
42 |
43 |
Username:
44 |
45 |
46 |
47 |
48 |
49 |
50 | Lesson Objective
51 |
52 |
At the end of this lesson, you will be able to:
53 |
54 |
Describe how to write commands for working with Delta tables in Databricks SQL
55 |
56 |
57 |
58 |
59 |
60 |
61 |
62 |
63 |
68 |
69 |
70 |
71 |
72 |
SELECT on Delta Tables
73 |
74 |
So far, the SQL commands we have used are generic to most flavors of SQL. In the next few queries, we are going to look at commands that are specific to using SELECT on Delta tables.
75 |
Delta tables keep a log of changes that we can view by running the command below.
76 |
77 |
Run the code below.
78 |
79 |
After running DESCRIBE HISTORY, we can see that we are on version number 0 and we can see a timestamp of when this change was made.
80 |
81 |
82 |
83 |
84 |
86 |
88 |
89 |
90 |
95 |
96 |
97 |
98 |
99 |
SELECT on Delta Tables -- Updating the Table
100 |
101 |
We are going to make a change to the table.
102 |
103 |
Run the code below.
104 |
105 |
The code uses an UPDATE statement to make a change to the table. We will be discussing UPDATE later on. For now, we just need to understand that a change was made to the table. We also reran our DESCRIBE HISTORY command, and note that we have a new version in the log, with a new timestamp.
106 |
107 |
108 |
109 |
110 |
113 |
116 |
117 |
118 |
123 |
124 |
125 |
126 |
127 |
SELECT on Delta Tables -- Updating the Table back to where it was
128 |
129 |
We are going to make a change to the table.
130 |
131 |
Run the code below.
132 |
133 |
The code uses an UPDATE statement to update the loyalty_segment back to its original value. We also reran our DESCRIBE HISTORY command, and note that we have a new version in the log, with a new timestamp.
134 |
135 |
136 |
137 |
138 |
141 |
144 |
145 |
146 |
151 |
152 |
153 |
154 |
155 |
SELECT on Delta Tables -- VERSION AS OF
156 |
157 |
We can now use a special predicate for use with Delta tables: VERSION AS OF
158 |
159 |
Run the code below.
160 |
161 |
By using VERSION AS OF, we can SELECT from specific versions of the table. This feature of Delta tables is called "Time Travel," and is very powerful.
162 |
We can also use TIMESTAMP AS OF to SELECTbased on a table's state on a specific date, and you can find more information in the documentation.
163 |
164 |
165 |
166 |
167 |
169 |
171 |
172 |
173 |
178 |
179 |
180 |
181 |
182 |
MERGE INTO
183 |
184 |
Certainly, there are times when we want to insert new data but ensure we don't re-insert matched data. This is where we use MERGE INTO. MERGE INTO will merge two tables together, but you specify in which column to look for matched data and what to do when a match is found. Let's run the code and examine the command in more detail.
185 |
186 |
Run the code below.
187 |
188 |
We are merging the source_suppliers table into the suppliers table. After ON keyword, we provide the columns on which we want to look for matches. We then state what the command should do when a match is found. In this example, we are inserting the row when the two columns are not matched. Thus, if the columns match, the row is ignored. Notice that the count is the exact same as the original table. This is because the two tables have the exact same data, since we overwrote the suppliers table with the source_suppliers table earlier in the lesson.
DAWD 02-5-1 - Demo - Delta Commands in Databricks SQL
37 |
The two fields below are used to customize queries used in this course. Enter your schema (database) name and username, and press "Enter" to populate necessary information in the queries on this page.
38 |
39 |
40 |
Schema Name:
41 |
42 |
43 |
Username:
44 |
45 |
46 |
47 |
48 |
49 |
50 | Lesson Objective
51 |
52 |
At the end of this lesson, you will be able to:
53 |
54 |
Describe how to write commands for working with Delta tables in Databricks SQL
55 |
56 |
57 |
58 |
59 |
60 |
61 |
62 |
63 |
68 |
69 |
70 |
71 |
72 |
SELECT on Delta Tables
73 |
74 |
So far, the SQL commands we have used are generic to most flavors of SQL. In the next few queries, we are going to look at commands that are specific to using SELECT on Delta tables.
75 |
Delta tables keep a log of changes that we can view by running the command below.
76 |
77 |
Run the code below.
78 |
79 |
After running DESCRIBE HISTORY, we can see that we are on version number 0 and we can see a timestamp of when this change was made.
80 |
81 |
82 |
83 |
84 |
86 |
88 |
89 |
90 |
95 |
96 |
97 |
98 |
99 |
SELECT on Delta Tables -- Updating the Table
100 |
101 |
We are going to make a change to the table.
102 |
103 |
Run the code below.
104 |
105 |
The code uses an UPDATE statement to make a change to the table. We will be discussing UPDATE later on. For now, we just need to understand that a change was made to the table. We also reran our DESCRIBE HISTORY command, and note that we have a new version in the log, with a new timestamp.
106 |
107 |
108 |
109 |
110 |
113 |
116 |
117 |
118 |
123 |
124 |
125 |
126 |
127 |
SELECT on Delta Tables -- Updating the Table back to where it was
128 |
129 |
We are going to make a change to the table.
130 |
131 |
Run the code below.
132 |
133 |
The code uses an UPDATE statement to update the loyalty_segment back to its original value. We also reran our DESCRIBE HISTORY command, and note that we have a new version in the log, with a new timestamp.
134 |
135 |
136 |
137 |
138 |
141 |
144 |
145 |
146 |
151 |
152 |
153 |
154 |
155 |
SELECT on Delta Tables -- VERSION AS OF
156 |
157 |
We can now use a special predicate for use with Delta tables: VERSION AS OF
158 |
159 |
Run the code below.
160 |
161 |
By using VERSION AS OF, we can SELECT from specific versions of the table. This feature of Delta tables is called "Time Travel," and is very powerful.
162 |
We can also use TIMESTAMP AS OF to SELECTbased on a table's state on a specific date, and you can find more information in the documentation.
163 |
164 |
165 |
166 |
167 |
169 |
171 |
172 |
173 |
178 |
179 |
180 |
181 |
182 |
MERGE INTO
183 |
184 |
Certainly, there are times when we want to insert new data but ensure we don't re-insert matched data. This is where we use MERGE INTO. MERGE INTO will merge two tables together, but you specify in which column to look for matched data and what to do when a match is found. Let's run the code and examine the command in more detail.
185 |
186 |
Run the code below.
187 |
188 |
We are merging the source_suppliers table into the suppliers table. After ON keyword, we provide the columns on which we want to look for matches. We then state what the command should do when a match is found. In this example, we are inserting the row when the two columns are not matched. Thus, if the columns match, the row is ignored. Notice that the count is the exact same as the original table. This is because the two tables have the exact same data, since we overwrote the suppliers table with the source_suppliers table earlier in the lesson.