Create table, concatenate data

I am trying to concatenate four fields from one table: org.address1,
org.city, org.state, org.zip with at least four fields from another table:
url.http_start, url.plus_sign, url.tail and url.http_end in a trigger that
would fire whenever a new record was created. The end stored result would be
an address used for Yahoo maps and would look like this:
http://maps.yahoo.com/py/maps.py?Pyt...et+Map=Get+Map
The data in the url table would have this data in it and really should just
be one row used over and over again.
url.http_start = http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr=
url.plus_sign = +
url.http_tail = &csz=
url.http_end = &Get+Map=Get+Map
I've come at this from a few different angles but I'm afraid my skills are
not up to what I am looking for; I would appreciate anyone's input.
I dont't unserstand the "plus_sign" thing, but I think you want something
like this. Since the tables are not related, you cannot really join on them
and you most likely want to store the "http" values in local variables:
DECLARE @.start varchar (100)
DECLARE @.plus char(1)
DECLARE @.tail varchar(100)
DECLARE @.end varchar(100)
SELECT @.start = http_start, @.plus = plus_sign, @.tail = http_tail, @.end =
http_end
FROM HTTP_TABLE
SELECT@.start + org.address1 + @.tail + org.city + @.plus + org.state + @.plus +
@.plus + org.zip + @.end
FROM ORG_TABLE org
Like I said, not sure what you wer going for with the "plus_sign" column,
but it looks like that might get you going on it
HTH,
John Scragg
"Steve1445" wrote:

> I am trying to concatenate four fields from one table: org.address1,
> org.city, org.state, org.zip with at least four fields from another table:
> url.http_start, url.plus_sign, url.tail and url.http_end in a trigger that
> would fire whenever a new record was created. The end stored result would be
> an address used for Yahoo maps and would look like this:
> http://maps.yahoo.com/py/maps.py?Pyt...et+Map=Get+Map
> The data in the url table would have this data in it and really should just
> be one row used over and over again.
> url.http_start = http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr=
> url.plus_sign = +
> url.http_tail = &csz=
> url.http_end = &Get+Map=Get+Map
> I've come at this from a few different angles but I'm afraid my skills are
> not up to what I am looking for; I would appreciate anyone's input.
>
|||Thanks for your reply, I was having troubles with the + sign in the final
concatenate being seen as something other than just what it is, a plus sign.
So I figured I would just store the actual plus sign in the plus_sign field
to use in the final concatenation. Thanks for your help. Steve
"John Scragg" wrote:
[vbcol=seagreen]
> I dont't unserstand the "plus_sign" thing, but I think you want something
> like this. Since the tables are not related, you cannot really join on them
> and you most likely want to store the "http" values in local variables:
> DECLARE @.start varchar (100)
> DECLARE @.plus char(1)
> DECLARE @.tail varchar(100)
> DECLARE @.end varchar(100)
> SELECT @.start = http_start, @.plus = plus_sign, @.tail = http_tail, @.end =
> http_end
> FROM HTTP_TABLE
> SELECT@.start + org.address1 + @.tail + org.city + @.plus + org.state + @.plus +
> @.plus + org.zip + @.end
> FROM ORG_TABLE org
> Like I said, not sure what you wer going for with the "plus_sign" column,
> but it looks like that might get you going on it
> HTH,
> John Scragg
> "Steve1445" wrote:

Create table, concatenate data

I am trying to concatenate four fields from one table: org.address1,
org.city, org.state, org.zip with at least four fields from another table:
url.http_start, url.plus_sign, url.tail and url.http_end in a trigger that
would fire whenever a new record was created. The end stored result would b
e
an address used for Yahoo maps and would look like this:
http://maps.yahoo.com/py/maps.py?Py...Get+Map=Get+Map
The data in the url table would have this data in it and really should just
be one row used over and over again.
url.http_start = http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr=
url.plus_sign = +
url.http_tail = &csz=
url.http_end = &Get+Map=Get+Map
I've come at this from a few different angles but I'm afraid my skills are
not up to what I am looking for; I would appreciate anyone's input.I dont't unserstand the "plus_sign" thing, but I think you want something
like this. Since the tables are not related, you cannot really join on them
and you most likely want to store the "http" values in local variables:
DECLARE @.start varchar (100)
DECLARE @.plus char(1)
DECLARE @.tail varchar(100)
DECLARE @.end varchar(100)
SELECT @.start = http_start, @.plus = plus_sign, @.tail = http_tail, @.end =
http_end
FROM HTTP_TABLE
SELECT@.start + org.address1 + @.tail + org.city + @.plus + org.state + @.plus +
@.plus + org.zip + @.end
FROM ORG_TABLE org
Like I said, not sure what you wer going for with the "plus_sign" column,
but it looks like that might get you going on it
HTH,
John Scragg
"Steve1445" wrote:

> I am trying to concatenate four fields from one table: org.address1,
> org.city, org.state, org.zip with at least four fields from another table:
> url.http_start, url.plus_sign, url.tail and url.http_end in a trigger that
> would fire whenever a new record was created. The end stored result would
be
> an address used for Yahoo maps and would look like this:
> http://maps.yahoo.com/py/maps.py?Py...Get+Map=Get+Map
> The data in the url table would have this data in it and really should jus
t
> be one row used over and over again.
> url.http_start = http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr=
> url.plus_sign = +
> url.http_tail = &csz=
> url.http_end = &Get+Map=Get+Map
> I've come at this from a few different angles but I'm afraid my skills are
> not up to what I am looking for; I would appreciate anyone's input.
>|||Thanks for your reply, I was having troubles with the + sign in the final
concatenate being seen as something other than just what it is, a plus sign.
So I figured I would just store the actual plus sign in the plus_sign field
to use in the final concatenation. Thanks for your help. Steve
"John Scragg" wrote:
[vbcol=seagreen]
> I dont't unserstand the "plus_sign" thing, but I think you want something
> like this. Since the tables are not related, you cannot really join on the
m
> and you most likely want to store the "http" values in local variables:
> DECLARE @.start varchar (100)
> DECLARE @.plus char(1)
> DECLARE @.tail varchar(100)
> DECLARE @.end varchar(100)
> SELECT @.start = http_start, @.plus = plus_sign, @.tail = http_tail, @.end =
> http_end
> FROM HTTP_TABLE
> SELECT@.start + org.address1 + @.tail + org.city + @.plus + org.state + @.plus
+
> @.plus + org.zip + @.end
> FROM ORG_TABLE org
> Like I said, not sure what you wer going for with the "plus_sign" column,
> but it looks like that might get you going on it
> HTH,
> John Scragg
> "Steve1445" wrote:
>

Create table, concatenate data

I am trying to concatenate four fields from one table: org.address1,
org.city, org.state, org.zip with at least four fields from another table:
url.http_start, url.plus_sign, url.tail and url.http_end in a trigger that
would fire whenever a new record was created. The end stored result would be
an address used for Yahoo maps and would look like this:
http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr=123+Main+Street&csz=Anywhere+KA++09186&Get+Map=Get+Map
The data in the url table would have this data in it and really should just
be one row used over and over again.
url.http_start = http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr= url.plus_sign = +
url.http_tail = &csz= url.http_end = &Get+Map=Get+Map
I've come at this from a few different angles but I'm afraid my skills are
not up to what I am looking for; I would appreciate anyone's input.I dont't unserstand the "plus_sign" thing, but I think you want something
like this. Since the tables are not related, you cannot really join on them
and you most likely want to store the "http" values in local variables:
DECLARE @.start varchar (100)
DECLARE @.plus char(1)
DECLARE @.tail varchar(100)
DECLARE @.end varchar(100)
SELECT @.start = http_start, @.plus = plus_sign, @.tail = http_tail, @.end =http_end
FROM HTTP_TABLE
SELECT@.start + org.address1 + @.tail + org.city + @.plus + org.state + @.plus +
@.plus + org.zip + @.end
FROM ORG_TABLE org
Like I said, not sure what you wer going for with the "plus_sign" column,
but it looks like that might get you going on it :)
HTH,
John Scragg
"Steve1445" wrote:
> I am trying to concatenate four fields from one table: org.address1,
> org.city, org.state, org.zip with at least four fields from another table:
> url.http_start, url.plus_sign, url.tail and url.http_end in a trigger that
> would fire whenever a new record was created. The end stored result would be
> an address used for Yahoo maps and would look like this:
> http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr=123+Main+Street&csz=Anywhere+KA++09186&Get+Map=Get+Map
> The data in the url table would have this data in it and really should just
> be one row used over and over again.
> url.http_start = http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr=
> url.plus_sign = +
> url.http_tail = &csz=> url.http_end = &Get+Map=Get+Map
> I've come at this from a few different angles but I'm afraid my skills are
> not up to what I am looking for; I would appreciate anyone's input.
>|||Thanks for your reply, I was having troubles with the + sign in the final
concatenate being seen as something other than just what it is, a plus sign.
So I figured I would just store the actual plus sign in the plus_sign field
to use in the final concatenation. Thanks for your help. Steve
"John Scragg" wrote:
> I dont't unserstand the "plus_sign" thing, but I think you want something
> like this. Since the tables are not related, you cannot really join on them
> and you most likely want to store the "http" values in local variables:
> DECLARE @.start varchar (100)
> DECLARE @.plus char(1)
> DECLARE @.tail varchar(100)
> DECLARE @.end varchar(100)
> SELECT @.start = http_start, @.plus = plus_sign, @.tail = http_tail, @.end => http_end
> FROM HTTP_TABLE
> SELECT@.start + org.address1 + @.tail + org.city + @.plus + org.state + @.plus +
> @.plus + org.zip + @.end
> FROM ORG_TABLE org
> Like I said, not sure what you wer going for with the "plus_sign" column,
> but it looks like that might get you going on it :)
> HTH,
> John Scragg
> "Steve1445" wrote:
> > I am trying to concatenate four fields from one table: org.address1,
> > org.city, org.state, org.zip with at least four fields from another table:
> > url.http_start, url.plus_sign, url.tail and url.http_end in a trigger that
> > would fire whenever a new record was created. The end stored result would be
> > an address used for Yahoo maps and would look like this:
> >
> > http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr=123+Main+Street&csz=Anywhere+KA++09186&Get+Map=Get+Map
> >
> > The data in the url table would have this data in it and really should just
> > be one row used over and over again.
> >
> > url.http_start = http://maps.yahoo.com/py/maps.py?Pyt=Tmap&addr=
> > url.plus_sign = +
> > url.http_tail = &csz=> > url.http_end = &Get+Map=Get+Map
> >
> > I've come at this from a few different angles but I'm afraid my skills are
> > not up to what I am looking for; I would appreciate anyone's input.
> >

Create File Option In File Connection Manager

May be it's too late, but I think this requests could be scheduled at least
for a SP1 if it's not possible for the RTM.

1) Execute Task without debugger: it would be very nice to be able to execute
a single task without going in debugging mode. Just as you would ask "Start
Without Debugging CTRL+F5" but for a single task

2) Customize default properties for task and component: when you drag a task
on the package you get a default value for the properties that you could want
to change; often I need to change the same property in the same way each time
(for example I'd like to set the Batch Size for a OLE DB destination to 1000
instead than 0)

3) If you open a package and connections to data source are not available,
propose to "work offline" at the first failed connection.

IMHO, these features would be very important for developer productivity.

Marco Russo
http://sqljunkies.com/weblog/sqlbi

Hi Marco,

All great suggestions. Can you open them in BetaPlace? Unfortunately they're too late for SQL Server 2005 but we'd love to revisit them for the future.

For #2, Copy/Paste might be a short term solution.

regards,
ash|||I cannot find the thread where someone from Microsoft solicited suggestions for changes; I thought it was in a thread by Jamie Thompson, but somehow I am now overlooking it (or misremembering).
In any case, in the hopes that someone relevant sees this, I have three more.
* In any editor for any component, have a visible indicator on all properties which are supplanted at run-time by expressions. For example, have the values in red. This is to indicate that what you are seeing is not what will be used.
* Mark all the boxes which have event handlers attached. As above, this is to inform the human that there is more here than is apparent, and that the human should go track down the "more" (in this case, event handlers), to really find out what is happening.
* Have a list, or tree view, of all the event handlers. I've not figured out anyway to find, say 20 event handlers scattered across 500 boxes in many packages, except by the slowly going through and double-clicking on every box looking for event handlers. This seems to me a terrible way to find event handlers; I don't know if I'm overlooking something obvious (I hope), but in case not, and perhaps in any case?, this request for enhancement.

(I cannot log in to betaplace; I spent some time trying to do so, and waving my mouse around clicking on invisible buttons, and I never got past a page saying that my account would be activated someday, I think, and I cannot even remember the sequence of steps to get there again now.)
|||Great ideas Perry, I second all of them. The one about indicating in the control-flow which tasks/containers have eventhandlers on them is inspired.

Your idea about a visual representation of which properties have expressions on them has already been raised. Hopefully we'll see it in the next version!

-Jamie|||Yes, I third them! In addition, it would be nice to see the ability to copy/paste/modify multiple variables. Managing variables and managing parent variables in package configurations is not easily done incurrent state, especially when you are dealing with 100+ packages all sharing same/similar variables.|||How about something that shows underlying execution plan (akin to query plan) for the entire package with cost weightage?

regards,
Nitesh|||If you've been using Integration Services and have some feedback for how to make it better, we'd love to hear more.

Please add to this thread what you'd like to see added, fixed, changed, tweeked, or removed from Integration Services.

Your feedback is valuable.
We can't promise we'll be able to make it all happen, but certainly the guidance you give here will influence planning for the next version of Integration services.

Thanks,
|||The biggest pains for me so far in designing our ETL for our warehouse have been:

- Reusing data flows, I am doing a hack that lets the data flows run over a set of tables, performing work on the common columns. What would be useful is if you can define a "table set" within SSIS and then bind a data flow to the table set (where the table set is limited to the columns/types common across all tables.) I don't know if this would have to fit into the foreach stuff, or if it would be all within the data flow itself.
- Working with tables with LOTs of columns. I have a table with about 200 columns or so that I need to do a slowly changing dimension transform on. I also need to write script components that output 200 columns for inserting into the table. The script task input/output dialog makes it painful to enter the variables one by one, and the SCD wizard makes it too painful to do it by hand, so I actually went into the XML itself and changed the stuff (carefully :)) Not sure how to address this, but another major thing that's probably more of an issue to fix is that the SCD component goes insanely slow when you double click on it if you have a whole lot of columns like me. (Takes a good 3-5 minutes to come up.)
- I posted a thread earlier, but to re-iterate -- since we can't reuse data flows most of the time nor script tasks, cut and pasting should be cleaned up a bit so the formatting doesn't get completely destroyed when you paste in a huge block of data flow/control flow tasks.
- Undo! :)
- Another small feature suggestion would be a more complex lookup task that had inherently a built in behavior for when the lookup fails. I have an "Unknown" member for each dimension, and if my lookup fails for a certain member of a fact table I need to link it to the Unknown member. What this translates to are a conditional split for if the key being looked up is NULL (or 0) and then setting it to zero if it was NULL or actually doing the lookup, and then doing a union of the rows again. I realize I could just rely upon the error output of the Lookup, but that seems broken to me since "Unknown" is an expected behavior. The ideal situation is for the Lookup Task to have an optional default value to use if the lookup fails and/or if the column being looked up is NULL.

|||Great suggestions!
Keep them coming!
K|||On the note of the Lookup Task, I think it's probably an extremely common use case where you have to translate a set of fact table business keys to surrogate dimension keys. (Project REAL, for example, seems to have a huge data flow to do this, and so do I.) With this in mind, it might be useful to have one single lookup task to translate all the keys (my current package has like 15 lookup tasks and a whole lot of conditionals for the aforementioned "Unknown" behavior checking.) Having one task that has a series of "table, join key, lookup value, lookup column, default value if null or not found" would consolidate my 40-50 tasks into a single one (which probably could internally do the lookups in parallel, increasing performance.)

|||

Ok here's my wish list,

1. Advanced Editor support for >1 input. (This should enable the script component with > 1 input)

2. Read only access to the whole package from componentmetadata, not just that related to the component.

3. Parallel For each loops. Performance.

4. Option on Raw file to create once per package. This allows the same raw destination to be used in a loop

5. Debug support for script component (not just the task)

6. Parallel multicast. Says it all really performance (I know the memory issue but it should be an option. Allows for the creation of a new execution tree. It would be great if the compiler (process that produces execution tree) could figure this out. This would probably need to now the distribution of data being processed.

7. Suggest Types for flat files to provide the option of reading a whole file. This is to avoid encountering bugs during run time, which is very time consuming.

8. Suggest types for flat files to all for data to be just strings, rather than convert data to proper types. This is for performance

9. IIS Log file connection both source and destination would be good. But would settle for source.

10. Multiple data readers out of package. This would enable a package to produce multiple summaries and have them consumed by a report or other application.

11. Be able to drag a connection from one component to another. Its a real pain to delete one connection to be recreate it to the other component. This looses any data viewers

That'll do for now.

|||Thanks Simon. Excellent input. Thanks!
Anyone else?
K|||

I would like to see 3 big key improvements within SSIS. I have raised this before, Kirk asked me to send him a mail, which I never got around to do it. Sorry Kirk.

1. Data Profiler. This is quite crucial when you analyse the data to determine how bad the data is etc. Yes I know, the feature is sort of there but it is not good enough. It need to be improved considerably. We should be able to put any type of files and profile it before we start the work. Saves lot of time. It should be quick and simple to do, in the meantime it should be powerful.

2. Meta Data Management Tool. This can be web based tool / something along those line, which can be given to the business users to indetify for example, how we derive Net Sales column in the fact table. From my own experience, spent hours / days explaining how we derive each column. In a huge data warehousing environment it is very time consuming. This is not fun, i rather be writing SSIS package instead .

3. Dependancy Analysis. I would like to see a tool that would do the dependancy analysis on the fly, if I specify, that I am going to drop column A, it should run some kind of routine and tells me if you drop this column from your SSIS package, it will affect this table, cube and package etc. Run the check against the metadata only, therefore it should be quick. Save lots of time and avoid mistakes happening.

These are my requests. I know they are big requests, but I think we do need them in Microsoft environment as other competitors got similar products.

What everyone else think about these features.

Thanks
Sutha

|||I've already fed alot of stuff back to Kirk offline but just for the edification of everyone else, here are some ideas:
http://blogs.conchango.com/jamiethomson/archive/2005/05/09/1398.aspx
http://blogs.conchango.com/jamiethomson/archive/2005/05/16/1419.aspx
http://blogs.conchango.com/jamiethomson/archive/2005/02/05/929.aspx
http://blogs.conchango.com/jamiethomson/archive/2005/05/26/1470.aspx
http://blogs.conchango.com/jamiethomson/archive/2005/09/07/2130.aspx

-Jamie|||Sometimes .dtsx files get corrupted. Don't know why...don't know how!

It would be useful to have a tool to analyse a corrupt .dtsx file to tell you exactly what's wrong with it, how to fix it, possibly even fix it for you etc.... The error messages you get when trying to load it aren't really useful.

-Jamie

Create File Option In File Connection Manager

May be it's too late, but I think this requests could be scheduled at least
for a SP1 if it's not possible for the RTM.

1) Execute Task without debugger: it would be very nice to be able to execute
a single task without going in debugging mode. Just as you would ask "Start
Without Debugging CTRL+F5" but for a single task

2) Customize default properties for task and component: when you drag a task
on the package you get a default value for the properties that you could want
to change; often I need to change the same property in the same way each time
(for example I'd like to set the Batch Size for a OLE DB destination to 1000
instead than 0)

3) If you open a package and connections to data source are not available,
propose to "work offline" at the first failed connection.

IMHO, these features would be very important for developer productivity.

Marco Russo
http://sqljunkies.com/weblog/sqlbi

Hi Marco,

All great suggestions. Can you open them in BetaPlace? Unfortunately they're too late for SQL Server 2005 but we'd love to revisit them for the future.

For #2, Copy/Paste might be a short term solution.

regards,
ash|||I cannot find the thread where someone from Microsoft solicited suggestions for changes; I thought it was in a thread by Jamie Thompson, but somehow I am now overlooking it (or misremembering).
In any case, in the hopes that someone relevant sees this, I have three more.
* In any editor for any component, have a visible indicator on all properties which are supplanted at run-time by expressions. For example, have the values in red. This is to indicate that what you are seeing is not what will be used.
* Mark all the boxes which have event handlers attached. As above, this is to inform the human that there is more here than is apparent, and that the human should go track down the "more" (in this case, event handlers), to really find out what is happening.
* Have a list, or tree view, of all the event handlers. I've not figured out anyway to find, say 20 event handlers scattered across 500 boxes in many packages, except by the slowly going through and double-clicking on every box looking for event handlers. This seems to me a terrible way to find event handlers; I don't know if I'm overlooking something obvious (I hope), but in case not, and perhaps in any case?, this request for enhancement.

(I cannot log in to betaplace; I spent some time trying to do so, and waving my mouse around clicking on invisible buttons, and I never got past a page saying that my account would be activated someday, I think, and I cannot even remember the sequence of steps to get there again now.)|||Great ideas Perry, I second all of them. The one about indicating in the control-flow which tasks/containers have eventhandlers on them is inspired.

Your idea about a visual representation of which properties have expressions on them has already been raised. Hopefully we'll see it in the next version!

-Jamie|||Yes, I third them! In addition, it would be nice to see the ability to copy/paste/modify multiple variables. Managing variables and managing parent variables in package configurations is not easily done incurrent state, especially when you are dealing with 100+ packages all sharing same/similar variables.|||How about something that shows underlying execution plan (akin to query plan) for the entire package with cost weightage?

regards,
Nitesh|||If you've been using Integration Services and have some feedback for how to make it better, we'd love to hear more.

Please add to this thread what you'd like to see added, fixed, changed, tweeked, or removed from Integration Services.

Your feedback is valuable.
We can't promise we'll be able to make it all happen, but certainly the guidance you give here will influence planning for the next version of Integration services.

Thanks,
|||The biggest pains for me so far in designing our ETL for our warehouse have been:

- Reusing data flows, I am doing a hack that lets the data flows run over a set of tables, performing work on the common columns. What would be useful is if you can define a "table set" within SSIS and then bind a data flow to the table set (where the table set is limited to the columns/types common across all tables.) I don't know if this would have to fit into the foreach stuff, or if it would be all within the data flow itself.
- Working with tables with LOTs of columns. I have a table with about 200 columns or so that I need to do a slowly changing dimension transform on. I also need to write script components that output 200 columns for inserting into the table. The script task input/output dialog makes it painful to enter the variables one by one, and the SCD wizard makes it too painful to do it by hand, so I actually went into the XML itself and changed the stuff (carefully :)) Not sure how to address this, but another major thing that's probably more of an issue to fix is that the SCD component goes insanely slow when you double click on it if you have a whole lot of columns like me. (Takes a good 3-5 minutes to come up.)
- I posted a thread earlier, but to re-iterate -- since we can't reuse data flows most of the time nor script tasks, cut and pasting should be cleaned up a bit so the formatting doesn't get completely destroyed when you paste in a huge block of data flow/control flow tasks.
- Undo! :)
- Another small feature suggestion would be a more complex lookup task that had inherently a built in behavior for when the lookup fails. I have an "Unknown" member for each dimension, and if my lookup fails for a certain member of a fact table I need to link it to the Unknown member. What this translates to are a conditional split for if the key being looked up is NULL (or 0) and then setting it to zero if it was NULL or actually doing the lookup, and then doing a union of the rows again. I realize I could just rely upon the error output of the Lookup, but that seems broken to me since "Unknown" is an expected behavior. The ideal situation is for the Lookup Task to have an optional default value to use if the lookup fails and/or if the column being looked up is NULL.|||Great suggestions!
Keep them coming!
K|||On the note of the Lookup Task, I think it's probably an extremely common use case where you have to translate a set of fact table business keys to surrogate dimension keys. (Project REAL, for example, seems to have a huge data flow to do this, and so do I.) With this in mind, it might be useful to have one single lookup task to translate all the keys (my current package has like 15 lookup tasks and a whole lot of conditionals for the aforementioned "Unknown" behavior checking.) Having one task that has a series of "table, join key, lookup value, lookup column, default value if null or not found" would consolidate my 40-50 tasks into a single one (which probably could internally do the lookups in parallel, increasing performance.)
|||

Ok here's my wish list,

1. Advanced Editor support for >1 input. (This should enable the script component with > 1 input)

2. Read only access to the whole package from componentmetadata, not just that related to the component.

3. Parallel For each loops. Performance.

4. Option on Raw file to create once per package. This allows the same raw destination to be used in a loop

5. Debug support for script component (not just the task)

6. Parallel multicast. Says it all really performance (I know the memory issue but it should be an option. Allows for the creation of a new execution tree. It would be great if the compiler (process that produces execution tree) could figure this out. This would probably need to now the distribution of data being processed.

7. Suggest Types for flat files to provide the option of reading a whole file. This is to avoid encountering bugs during run time, which is very time consuming.

8. Suggest types for flat files to all for data to be just strings, rather than convert data to proper types. This is for performance

9. IIS Log file connection both source and destination would be good. But would settle for source.

10. Multiple data readers out of package. This would enable a package to produce multiple summaries and have them consumed by a report or other application.

11. Be able to drag a connection from one component to another. Its a real pain to delete one connection to be recreate it to the other component. This looses any data viewers

That'll do for now.

|||Thanks Simon. Excellent input. Thanks!
Anyone else?
K|||

I would like to see 3 big key improvements within SSIS. I have raised this before, Kirk asked me to send him a mail, which I never got around to do it. Sorry Kirk.

1. Data Profiler. This is quite crucial when you analyse the data to determine how bad the data is etc. Yes I know, the feature is sort of there but it is not good enough. It need to be improved considerably. We should be able to put any type of files and profile it before we start the work. Saves lot of time. It should be quick and simple to do, in the meantime it should be powerful.

2. Meta Data Management Tool. This can be web based tool / something along those line, which can be given to the business users to indetify for example, how we derive Net Sales column in the fact table. From my own experience, spent hours / days explaining how we derive each column. In a huge data warehousing environment it is very time consuming. This is not fun, i rather be writing SSIS package instead .

3. Dependancy Analysis. I would like to see a tool that would do the dependancy analysis on the fly, if I specify, that I am going to drop column A, it should run some kind of routine and tells me if you drop this column from your SSIS package, it will affect this table, cube and package etc. Run the check against the metadata only, therefore it should be quick. Save lots of time and avoid mistakes happening.

These are my requests. I know they are big requests, but I think we do need them in Microsoft environment as other competitors got similar products.

What everyone else think about these features.

Thanks
Sutha

|||I've already fed alot of stuff back to Kirk offline but just for the edification of everyone else, here are some ideas:
http://blogs.conchango.com/jamiethomson/archive/2005/05/09/1398.aspx
http://blogs.conchango.com/jamiethomson/archive/2005/05/16/1419.aspx
http://blogs.conchango.com/jamiethomson/archive/2005/02/05/929.aspx
http://blogs.conchango.com/jamiethomson/archive/2005/05/26/1470.aspx
http://blogs.conchango.com/jamiethomson/archive/2005/09/07/2130.aspx

-Jamie|||Sometimes .dtsx files get corrupted. Don't know why...don't know how!

It would be useful to have a tool to analyse a corrupt .dtsx file to tell you exactly what's wrong with it, how to fix it, possibly even fix it for you etc.... The error messages you get when trying to load it aren't really useful.

-Jamie

Create Database Failed - Primary file must be at least 3 MB ...

I am trying to create a database with the following command

CREATE DATABASE [db1] ON PRIMARY
( NAME = N'db1', FILENAME = N'C:\Databases\Main\db1.mdf' , SIZE = 2048KB , FILEGROWTH = 1024KB )
LOG ON
( NAME = N'db1_log', FILENAME = N'C:\Databases\Main\db1_log.ldf' , SIZE = 1024KB , FILEGROWTH = 10%)

However, I always get the following error:

CREATE DATABASE failed. Primary file must be at least 3 MB to accommodate a copy of the model database.

I don't have a clue what's going wrong? The very same command is working fine on other machines. Its only one particular machine where it fails with the error message. Strangely enough, it worked on the "problem machine" too until I had to format the hard disk of the machine and re-install everything from scratch.

Any idea what could be going wrong? Please note that I am using SQL Server 2005.

On the server where this fails, verify the size of the Model database. I suspect it has inadvertently become larger than expected.

All newly created databases use the Model database as a 'template'. The new databases will start out no smaller than the Model database.

As this message indicates, since the Model database is about 3 MB in size, you MUST have a minimum size of 3 MB.

And you can reduce the size of the Model database. See Books Online, Topics: DBCC Shrinkdatabase, DBCC Shrinkfile

|||Thanks for your response. I checked the size of model database and found that it has a size of 3MB on that particular server as you said.
Actually I am creating a installation package which has to run on different machines. I am new to SQL Server so have a lesser understanding of many features of SQL Server.
Since, size of model database may just vary from machines to machines, I decided to entirely remove the SIZE=2048KB option from the CREATE DATABASE command. I tried it on my test machines and seems to work fine.
It seems that removing the SIZE option should not be a problem as system seems to pick up the size of model database on the server where is is executing.
Please correct if I am wrong or you see any negative impacts of this.
|||

You are correct, removing the [SIZE] parameter will ensure that you don't run into a similar problem.

I would suggest that immediately after the CREATE DATABASE statement, if your needs dictate a certain size to start with, that your code check the defined size and issue a ALTER DATABASE statement to increase if necessary.

|||I do not have any such requirement as of now so I might not be needing an 'alter database'
Its just that I am new to SQL server and I exported this script through SQL Server management studio. I wasn't aware that SIZE has a dependency on model database.

Anyway, thanks a lot for your suggestions.