Speeding up SQL transfer over a network

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP


Speeding up SQL transfer over a network



I have two SQL environments, data warehouse which collects data and a datamart which people access for a subset of the data, each with their own SQL 2016 databases. I run a script which pulls out data, transforms it and transfer it from the data warehouse to the datamart using Linked Servers. The entire process takes around 60+ hours to run. I want to avoid at all costs at having the data warehouse data in the datamart.



I experimented to see why the whole process was taking so long. I did a backup of the data warehouse, restored it onto the datamart and ran the import script and the entire process took around 3 hours to run. The script itself to 1.5 hours, telling me of the 60+ hours its the linked server transfer of data between the two servers that is the slowest part. I've pretty much ruled out network speed or issues between the two servers; this is all SQL. I'm trying to avoid having to write an application to do all of this in .NET if I can keep it in SQL.



Does anyone have any suggestions on how to improve performance time between SQL transfers?









By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

-

Popular posts from this blog

C# - How to create a semi transparent or blurred backcolor on windows form

Image
Clash Royale CLAN TAG #URR8PPP C# - How to create a semi transparent or blurred backcolor on windows form I am creating a Speed Meter to measure my internet speed and stays at top of every app, then I how do I make Its background semi transparent or blurred something like this. You can make ts semi opaque, however i'm not sure you'll easily be able to blur the background – TheGeneral 6 hours ago thanks ! then how do I make it semi opaque – Arwin Chua 6 hours ago Form.Opacity Property – TheGeneral 6 hours ago See th
Read more

Will Oldham

Image
"Palace Music" redirects here. For the racehorse, see Palace Music (horse). Bonnie 'Prince' Billy Will Oldham, June 6, 2009 Background information Birth name William Oldham Also known as Palace Brothers Palace Music Bonnie 'Prince' Billy Born ( 1970-12-24 ) December 24, 1970 (age 47) Louisville, Kentucky, United States Genres Folk alternative country country freak folk [1] Years active 1993–present Labels Drag City, Domino, Spunk Associated acts Dawn McCarthy, The Cairo Gang, Mekons, Matt Sweeney, Mick Turner, Tortoise, Trembling Bells, Harem Scarem, The Picket Line, Alex Neilson, Björk, Slint Website www.bonnieprincebilly.com Will Oldham (born December 24, 1970), better known by the stage name Bonnie 'Prince' Billy , is an American singer-songwriter and actor. From 1993 to 1997, he performed and recorded under variations of the Palace name, including the Palace Brothers, Palace Songs, and Palace Music. After releasing material under his own name,
Read more

Makefile test if variable is not empty

Makefile test if variable is not empty In a makefile I'm trying to I've created this simplified makefile to demonstrate my problem. Neither make a or make b executes the body of the if, I don't understand why not. make a make b .PHONY: a b a: $(eval MY_VAR = $(shell echo whatever)) @echo MY_VAR is $(MY_VAR) $(info $(MY_VAR)) ifneq ($(strip $(MY_VAR)),) @echo "should be executed" endif @echo done b: $(eval MY_VAR = $(shell echo '')) @echo MY_VAR is $(MY_VAR) $(info $(MY_VAR)) ifneq ($(strip $(MY_VAR)),) @echo "should not be executed" endif @echo done I'm using $ make --version GNU Make 3.81 There are several problems, and misunderstandings. Your MY_VAR is a make variable, set it without a <tab> to its value, and no need to $(eval ...) . MY_VAR:=$(shell echo whatever) is enough. No <tab> before $(info ...) neither, this is make – Zelnes
Read more