블로그 이미지
LifeisSimple

calendar

1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31

Notice

2011. 6. 16. 13:50 Brain Trainning/DataBase

인덱스 관련해서 반드시 알아야 하는.... 

출처 : http://www.databasejournal.com/features/mssql/sql-server-index-t-sql-statements.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+DatabaseJournalNews+%28Database+Journal+News%29&utm_content=Google+International

June 15, 2011

Top 10 Index T-SQL Statements That SQL Server DBAs Should Know

By Muthusamy Anantha Kumar aka The MAK

SQL Server DBAs know that indexes in databases are very similar to indexes in libraries. An index in a database is a structure associated with a table or view that speeds retrieval of rows from the table or view.

This article lists the top index-related T-SQL statements that are useful for SQL Server database administrators. The T-SQL statements covered in this article are classified into three categories: Index Definition or Create, Query — Query index related information and Maintenance.

Definition - Create Index

1. Clustered Index

Clustered indexes store the data rows in sorted order in the table based on their key values. Only one clustered index can be created per table, because the data rows themselves can only be sorted in one order.

A clustered index can be created while creating constraints like primary key on an existing table. Example:


ALTER TABLE [MyAddress]
ADD CONSTRAINT [PK_Address_AddressID] PRIMARY KEY CLUSTERED
(
[AddressID] ASC
) ON [PRIMARY]
GO

A clustered index can also be created on a column with no constraints related clause. Example:


CREATE CLUSTERED INDEX [MyAddress_id_CIX] ON [MyAddress1]
(
[ID] ASC
)ON [PRIMARY]
GO

2. Non Clustered Index

Generally, nonclustered indexes are created to improve the performance of frequently used queries not covered by the clustered index. In a nonclustered index, the logical order of the index does not match the physical stored order of the rows on disk.

A nonclustered Index can be created on an existing table covering columns not covered by clustered index. Example:


CREATE UNIQUE NONCLUSTERED INDEX
[NIX_col5_col2_col3_col4_col6]
ON [MyAddress]
(
[AddressLine1] ASC,
[AddressLine2] ASC,
[City] ASC,
[StateProvinceID] ASC,
[PostalCode] ASC
)ON [PRIMARY]
GO

A nonclustered index can also be created while creating constraints on the existing table. Example:


ALTER TABLE [MyAddressType]
ADD CONSTRAINT [DEFF_MyAddressType_ModifiedDate]
DEFAULT (getdate()) FOR [ModifiedDate]
GO

3. XML Index

An XML index can be created on an XML column and the table must have a clustered index on the primary key. The XML index can be primary or secondary.

A primary XML index can be created as shown below:


CREATE PRIMARY XML INDEX idx_xCol_MyTable on MyTable (xCol)

A secondary XML index can be created as shown below:


CREATE TABLE MyTable (Col1 INT PRIMARY KEY, XmlCol XML)
GO
-- Create primary index.
CREATE PRIMARY XML INDEX PIdx_MyTable_XmlCol
ON T(XmlCol)
GO
-- Create secondary indexes (PATH, VALUE, PROPERTY).
CREATE XML INDEX PIdx_MyTable_XmlCol_PATH ON MyTable(XmlCol)
USING XML INDEX PIdx_MyTable_XmlCol
FOR PATH
GO
CREATE XML INDEX PIdx_MyTable_XmlCol_VALUE ON T(XmlCol)
USING XML INDEX PIdx_MyTable_XmlCol
FOR VALUE
GO

4. Spatial Index

SQL Server 2008 provided a special type of column called a spatial column, which is a table column that contains data of a spatial data type, such as geometry or geography.

A spatial index can be created using the following syntax:


CREATE TABLE MySpatialTable(id int primary key, geometry_col geometry);
CREATE SPATIAL INDEX SIndx_MySpatialTable_geometry_col1
ON MySpatialTable(geometry_col)
WITH ( BOUNDING_BOX = ( 0, 0, 500, 200 ) );

Query Index related metadata

5. Find all Indexes

The following query can be used to query all the tables, columns and indexes on the current database:


SELECT OBJECT_SCHEMA_NAME(BaseT.[object_id],DB_ID()) AS [Schema],
BaseT.[name] AS [table_name], I.[name] AS [index_name], AC.[name] AS [column_name],
I.[type_desc]
FROM sys.[tables] AS BaseT
INNER JOIN sys.[indexes] I ON BaseT.[object_id] = I.[object_id]
INNER JOIN sys.[index_columns] IC ON I.[object_id] = IC.[object_id]
INNER JOIN sys.[all_columns] AC ON BaseT.[object_id] = AC.[object_id] AND IC.[column_id] = AC.[column_id]
WHERE BaseT.[is_ms_shipped] = 0 AND I.[type_desc] <> 'HEAP'
ORDER BY BaseT.[name], I.[index_id], IC.[key_ordinal]

6. Fragmentation

The following query can be used to find the index fragmentation on all the tables in the current database:


SELECT object_name(IPS.object_id) AS [TableName],
SI.name AS [IndexName],
IPS.Index_type_desc,
IPS.avg_fragmentation_in_percent,
IPS.avg_fragment_size_in_pages,
IPS.avg_page_space_used_in_percent,
IPS.record_count,
IPS.ghost_record_count,
IPS.fragment_count,
IPS.avg_fragment_size_in_pages
FROM sys.dm_db_index_physical_stats(db_id(DB_NAME()), NULL, NULL, NULL , 'DETAILED') IPS
JOIN sys.tables ST WITH (nolock) ON IPS.object_id = ST.object_id
JOIN sys.indexes SI WITH (nolock) ON IPS.object_id = SI.object_id AND IPS.index_id = SI.index_id
WHERE ST.is_ms_shipped = 0
order by IPS.avg_fragment_size_in_pages desc

7. Missing index

SQL Server keeps track of the indexes that it thinks you should create that will help in improving the performance of queries. The following query list all missing indexes.


SELECT sys.objects.name
, (avg_total_user_cost * avg_user_impact) * (user_seeks + user_scans) AS Impact
, 'CREATE NONCLUSTERED INDEX ix_IndexName ON ' + sys.objects.name COLLATE DATABASE_DEFAULT + ' ( ' + IsNull(mid.equality_columns, '') + CASE WHEN mid.inequality_columns IS NULL
THEN ''
ELSE CASE WHEN mid.equality_columns IS NULL
THEN ''
ELSE ',' END + mid.inequality_columns END + ' ) ' + CASE WHEN mid.included_columns IS NULL
THEN ''
ELSE 'INCLUDE (' + mid.included_columns + ')' END + ';' AS CreateIndexStatement
, mid.equality_columns
, mid.inequality_columns
, mid.included_columns
FROM sys.dm_db_missing_index_group_stats AS migs
INNER JOIN sys.dm_db_missing_index_groups AS mig ON migs.group_handle = mig.index_group_handle
INNER JOIN sys.dm_db_missing_index_details AS mid ON mig.index_handle = mid.index_handle AND mid.database_id = DB_ID()
INNER JOIN sys.objects WITH (nolock) ON mid.OBJECT_ID = sys.objects.OBJECT_ID
WHERE (migs.group_handle IN
(
SELECT TOP (500) group_handle
FROM sys.dm_db_missing_index_group_stats WITH (nolock)
ORDER BY (avg_total_user_cost * avg_user_impact) * (user_seeks + user_scans) DESC))
AND OBJECTPROPERTY(sys.objects.OBJECT_ID, 'isusertable')=1
ORDER BY 2 DESC , 3 DESC

8. Unused index

The following statement lists all the indexes that have not been used. This also generates the DROP index statement which can come handy when deleting the indexes.


SELECT o.name, indexname=i.name, i.index_id
, reads=user_seeks + user_scans + user_lookups
, writes = user_updates
, rows = (SELECT SUM(p.rows) FROM sys.partitions p WHERE p.index_id = s.index_id AND s.object_id = p.object_id)
, CASE
WHEN s.user_updates < 1 THEN 100
ELSE 1.00 * (s.user_seeks + s.user_scans + s.user_lookups) / s.user_updates
END AS reads_per_write
, 'DROP INDEX ' + QUOTENAME(i.name)
+ ' ON ' + QUOTENAME(c.name) + '.' + QUOTENAME(OBJECT_NAME(s.object_id)) as 'drop statement'
FROM sys.dm_db_index_usage_stats s
INNER JOIN sys.indexes i ON i.index_id = s.index_id AND s.object_id = i.object_id
INNER JOIN sys.objects o on s.object_id = o.object_id
INNER JOIN sys.schemas c on o.schema_id = c.schema_id
WHERE OBJECTPROPERTY(s.object_id,'IsUserTable') = 1
AND s.database_id = DB_ID()
AND i.type_desc = 'nonclustered'
AND i.is_primary_key = 0
AND i.is_unique_constraint = 0
AND (SELECT SUM(p.rows) FROM sys.partitions p WHERE p.index_id = s.index_id AND s.object_id = p.object_id) > 10000
ORDER BY reads

Index Maintainenance

9. Rebuild index

When an index gets fragmented, it requires defragmentation. Defragmentation can be done using the rebuild clause when altering a table. This command is equivalent to DBCC DBREINDEX in SQL Server versions prior to 2005. The command that can be used to rebuild the index is as follows:


USE AdventureWorks2008R2;
GO
ALTER INDEX PK_Employee_BusinessEntityID ON HumanResources.Employee
REBUILD;
GO

If ALL is not specified in rebuild, it will not rebuild a nonclustered index.

10. REORGANIZE index

Specifies that the index leaf level will be reorganized. The REORGANIZE statement is always performed online. This command is equivalent to DBCC INDEXDEFRAG in SQL Server versions prior to 2005.


USE AdventureWorks2008R2;
GO
ALTER INDEX PK_ProductPhoto_ProductPhotoID ON Production.ProductPhoto
REORGANIZE ;
GO

Conclusion

I hope you've found useful this list of the top Index-related T-SQL statements that are useful for SQL Server database administrators.

For more on T-SQL programming, see our T-SQL programming series

See all articles by MAK

posted by LifeisSimple
2011. 6. 14. 12:25 Brain Trainning/DataBase
GeoIP용 쿼리 

AMD Opteron 2.4G 에서 70번 정도 수행
(StartIPNum에 Unique 하지 않은 값이 있을 수 있으므로 문제가 될 수 있음)

declare @IPNum bigint

 

select @IPNum = convert(bigint, dbo.UFConvertIP('X', '180.70.93.57'));

 

with IPS as (

select top 5 endipnum, nationCD from dbo.T_GeoIPCountryWhois with (nolock) where @IPNum > startipnum order by startipnum desc

)

select top 1 nationCD

from IPS

where endipnum > @IPNum order by endipnum

posted by LifeisSimple
2011. 6. 13. 17:50 Brain Trainning/DataBase
출처 : http://www.sql-server-performance.com/2011/monitor-sql-server-replication-jobs/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+SQLServerPerformance+%28Latest+SQL-Server-Performance.com+Updates%29&utm_content=Google+International

Monitor SQL Server Replication Jobs

The Replication infrastructure in SQL Server is implemented using SQL Server Agent to execute the various components involved in the form of a job (e.g. LogReader agent job, Distribution agent job, Merge agent job)

SQL Server jobs execute a binary executable file which is basically C++ code.

You can download all the scripts for this article here

SQL Server Job Schedules

By default each of job has only one schedule that is set to Start automatically when SQL Server Agent starts. This schedule ensures that when ever the SQL Server Agent service is started all the replication components are also put into action. This is OK and makes sense but there is one problem with this default configuration that needs improvement  -  if for any reason one of the components fails it remains down in a stopped state.  

Unless you monitor the status of each component you will typically get to know about such a failure from a customer complaint as a result of missing data or data that is not up to date at the subscriber level. Furthermore, having any of these components in a stopped state can lead to more severe problems if not corrected within a short time.

The action required to improve on this default settings is in fact very simple. Adding a second schedule that is set as a Daily Reoccurring schedule which runs every 1 minute does the trick. SQL Server Agent’s scheduler module knows how to handle overlapping schedules so if the job is already being executed by another schedule it will not get executed again at the same time. So, in the event of a failure the failed job remains down for at most 60 seconds.

Many DBAs are not aware of this capability and so search for more complex solutions such as having an additional dedicated job running an external code in VBS or another scripting language that detects replication jobs in a stopped state and starts them but there is no need to seek such external solutions when what is needed can be accomplished by T-SQL code.

SQL Server Jobs Status

In addition to the 1 minute schedule we also want to ensure that key components in the replication are enabled so I can search for those components by their Category, and set their status to enabled in case they are disabled, by executing the stored procedure MonitorEnableReplicationAgents.

The jobs that I typically have handled are listed below but you may want to extend this, so below is the query to return all jobs along with their category.

SELECT category_id, name FROM msdb.dbo.syscategories ORDER BY category_id;

  • Distribution Cleanup
  • LogReader Agent
  • Distribution Agent

Snapshot Agent Jobs

By default when a publication is created, a snapshot agent job also gets created with a daily schedule. I see more organizations where the snapshot agent job does not need to be executed automatically by the SQL Server Agent  scheduler than organizations who   need a new snapshot generated automatically. To assure this setting is in place I created the stored procedure MonitorSnapshotAgentsSchedules which disables snapshot agent jobs and also deletes the job schedule.

It is worth mentioning that when the publication property immediate_sync is turned off then the snapshot files are not created when the Snapshot agent is executed by the job. You control this property when the publication is created with a parameter called @immediate_sync passed to sp_addpublication and for an existing publication you can use sp_changepublication.

Implementation

The scripts assume the existence of a database named PerfDB.

Steps:

  1. Run the scripts to create the stored procedures in the PerfDB database.
  2. Create a job that executes the stored procedures every hour.

-- Verify that the 1_Minute schedule exists.
EXEC PerfDB.dbo.MonitorReplicationAgentsSchedules @CategoryId = 10; /* Distribution */
EXEC PerfDB.dbo.MonitorReplicationAgentsSchedules @CategoryId = 13; /* LogReader     */
-- Verify all replication agents are enabled.
EXEC PerfDB.dbo.MonitorEnableReplicationAgents @CategoryId = 10; /* Distribution */
EXEC PerfDB.dbo.MonitorEnableReplicationAgents @CategoryId = 13; /* LogReader   */
EXEC PerfDB.dbo.MonitorEnableReplicationAgents @CategoryId = 11; /* Distribution clean up   */
-- Verify that Snapshot agents are disabled and have no schedule
EXEC PerfDB.dbo.MonitorSnapshotAgentsSchedules;

Want to read more of about replication? Check at my replication posts at my blog.

posted by LifeisSimple
2011. 5. 11. 17:39 Brain Trainning/Server

출처 : http://msdn.microsoft.com/ko-kr/library/ms176105
 

OBJECTPROPERTY(Transact-SQL)

SQL Server 2008 R2

현재 데이터베이스의 스키마 범위 개체에 대한 정보를 반환합니다. 스키마 범위 개체의 목록을 보려면 sys.objects(Transact-SQL)를 참조하십시오. DDL(데이터 정의 언어) 트리거 및 이벤트 알림과 같이 스키마 범위가 아닌 개체에 대해서는 이 함수를 사용할 수 없습니다.

항목 링크 아이콘 Transact-SQL 구문 표기 규칙

OBJECTPROPERTY ( id , property ) 
id

현재 데이터베이스에 있는 개체의 ID를 나타내는 식입니다. id는 int이며 현재 데이터베이스 컨텍스트에서 스키마 범위 개체로 간주됩니다.

property

id로 지정한 개체에 대해 반환될 정보를 나타내는 식입니다. property는 다음 값 중 하나일 수 있습니다.

참고 참고

달리 언급하지 않는 한 property가 유효한 속성 이름이 아니거나, id가 유효한 개체 ID가 아니거나, id가 지정된 property에 대해 지원되지 않는 개체 형식이거나, 호출자가 개체의 메타데이터를 볼 수 있는 권한이 없는 경우에는 NULL이 반환됩니다.

속성 이름

개체 형식

설명 및 반환 값

CnstIsClustKey

제약 조건

클러스터형 인덱스가 있는 PRIMARY KEY 제약 조건

1 = True

0 = False

CnstIsColumn

제약 조건

단일 열에 대한 CHECK, DEFAULT 또는 FOREIGN KEY 제약 조건

1 = True

0 = False

CnstIsDeleteCascade

제약 조건

ON DELETE CASCADE 옵션이 지정된 FOREIGN KEY 제약 조건

1 = True

0 = False

CnstIsDisabled

제약 조건

비활성화된 제약 조건입니다.

1 = True

0 = False

CnstIsNonclustKey

제약 조건

비클러스터형 인덱스가 있는 PRIMARY KEY 또는 UNIQUE 제약 조건입니다.

1 = True

0 = False

CnstIsNotRepl

제약 조건

NOT FOR REPLICATION 키워드를 사용하여 제약 조건을 정의합니다.

1 = True

0 = False

CnstIsNotTrusted

제약 조건

기존 행을 검사하지 않고 제약 조건을 사용했으므로 제약 조건이 모든 행에 적용되지 않을 수도 있습니다.

1 = True

0 = False

CnstIsUpdateCascade

제약 조건

ON UPDATE CASCADE 옵션이 지정된 FOREIGN KEY 제약 조건

1 = True

0 = False

ExecIsAfterTrigger

트리거

AFTER 트리거입니다.

1 = True

0 = False

ExecIsAnsiNullsOn

Transact-SQL 함수, Transact-SQL 프로시저, Transact-SQL 트리거, 뷰

만든 시간의 ANSI_NULLS 설정입니다.

1 = True

0 = False

ExecIsDeleteTrigger

트리거

DELETE 트리거입니다.

1 = True

0 = False

ExecIsFirstDeleteTrigger

트리거

테이블에 대해 DELETE가 실행될 때 처음으로 실행되는 트리거입니다.

1 = True

0 = False

ExecIsFirstInsertTrigger

트리거

테이블에 대해 INSERT가 실행될 때 처음으로 실행되는 트리거입니다.

1 = True

0 = False

ExecIsFirstUpdateTrigger

트리거

테이블에 대해 UPDATE가 실행될 때 처음으로 실행되는 트리거입니다.

1 = True

0 = False

ExecIsInsertTrigger

트리거

INSERT 트리거입니다.

1 = True

0 = False

ExecIsInsteadOfTrigger

트리거

INSTEAD OF 트리거입니다.

1 = True

0 = False

ExecIsLastDeleteTrigger

트리거

테이블에 대해 DELETE가 실행될 때 마지막으로 실행되는 트리거입니다.

1 = True

0 = False

ExecIsLastInsertTrigger

트리거

테이블에 대해 INSERT가 실행될 때 마지막으로 실행되는 트리거입니다.

1 = True

0 = False

ExecIsLastUpdateTrigger

트리거

테이블에 대해 UPDATE가 실행될 때 마지막으로 실행되는 트리거입니다.

1 = True

0 = False

ExecIsQuotedIdentOn

Transact-SQL 함수, Transact-SQL 프로시저, Transact-SQL 트리거, 뷰

생성 시의 QUOTED_IDENTIFIER 설정

1 = True

0 = False

ExecIsStartup

프로시저

시작 프로시저입니다.

1 = True

0 = False

ExecIsTriggerDisabled

트리거

비활성화된 트리거입니다.

1 = True

0 = False

ExecIsTriggerNotForRepl

트리거

NOT FOR REPLICATION으로 정의된 트리거입니다.

1 = True

0 = False

ExecIsUpdateTrigger

트리거

UPDATE 트리거입니다.

1 = True

0 = False

HasAfterTrigger

테이블, 뷰

테이블이나 뷰에 AFTER 트리거가 있습니다.

1 = True

0 = False

HasDeleteTrigger

테이블, 뷰

테이블이나 뷰에 DELETE 트리거가 있습니다.

1 = True

0 = False

HasInsertTrigger

테이블, 뷰

테이블이나 뷰에 INSERT 트리거가 있습니다.

1 = True

0 = False

HasInsteadOfTrigger

테이블, 뷰

테이블이나 뷰에 INSTEAD OF 트리거가 있습니다.

1 = True

0 = False

HasUpdateTrigger

테이블, 뷰

테이블이나 뷰에 UPDATE 트리거가 있습니다.

1 = True

0 = False

IsAnsiNullsOn

Transact-SQL 함수, Transact-SQL 프로시저, 테이블, Transact-SQL 트리거, 뷰

테이블에 대한 ANSI NULLS 옵션을 ON으로 지정합니다. 이는 Null 값에 대한 모든 비교가 UNKNOWN으로 계산된다는 의미입니다. 이 설정은 테이블이 존재하는 한 계산 열과 제약 조건을 포함하여 테이블 정의 내의 모든 식에 적용됩니다.

1 = True

0 = False

IsCheckCnst

임의의 스키마 범위 개체

CHECK 제약 조건입니다.

1 = True

0 = False

IsConstraint

임의의 스키마 범위 개체

열 또는 테이블에 대한 단일 열 CHECK, DEFAULT 또는 FOREIGN KEY 제약 조건입니다.

1 = True

0 = False

IsDefault

임의의 스키마 범위 개체

바운드 기본값입니다.

1 = True

0 = False

IsDefaultCnst

임의의 스키마 범위 개체

DEFAULT 제약 조건입니다.

1 = True

0 = False

IsDeterministic

함수, 뷰

함수 또는 뷰의 결정성 속성

1 = 결정적

0 = 비결정적

IsEncrypted

Transact-SQL 함수, Transact-SQL 프로시저, 테이블, Transact-SQL 트리거, 뷰

모듈 문의 원본 텍스트가 난독 처리된 형식으로 변환되었음을 나타냅니다. 난독 처리된 출력은 SQL Server 2005의 카탈로그 뷰 어디에서도 직접 표시되지 않습니다. 시스템 테이블 또는 데이터베이스 파일에 대한 액세스 권한이 없는 사용자는 난독 처리된 텍스트를 검색할 수 없습니다. 그러나 DAC 포트를 통해 시스템 테이블에 액세스하거나 데이터베이스 파일에 직접 액세스할 수 있는 사용자는 난독 처리된 텍스트를 사용할 수 있습니다. 또한 디버거를 서버 프로세스에 연결할 수 있는 사용자는 런타임에 메모리에서 원래 프로시저를 검색할 수 있습니다.

1 = 암호화됨

0 = 암호화되지 않음

기본 데이터 형식: int

IsExecuted

임의의 스키마 범위 개체

개체를 실행할 수 있습니다(뷰, 프로시저, 함수 또는 트리거).

1 = True

0 = False

IsExtendedProc

임의의 스키마 범위 개체

확장 프로시저입니다.

1 = True

0 = False

IsForeignKey

임의의 스키마 범위 개체

FOREIGN KEY 제약 조건입니다.

1 = True

0 = False

IsIndexed

테이블, 뷰

인덱스가 있는 테이블 또는 뷰입니다.

1 = True

0 = False

IsIndexable

테이블, 뷰

인덱스를 만들 수 있는 테이블 또는 뷰입니다.

1 = True

0 = False

IsInlineFunction

함수

인라인 함수입니다.

1 = 인라인 함수

0 = 비인라인 함수

IsMSShipped

임의의 스키마 범위 개체

SQL Server를 설치하는 동안 만들어진 개체입니다.

1 = True

0 = False

IsPrimaryKey

임의의 스키마 범위 개체

PRIMARY KEY 제약 조건입니다.

1 = True

0 = False

NULL = 함수가 아니거나 개체 ID가 유효하지 않습니다.

IsProcedure

임의의 스키마 범위 개체

프로시저입니다.

1 = True

0 = False

IsQuotedIdentOn

Transact-SQL 함수, Transact-SQL 프로시저, 테이블, Transact-SQL 트리거, 뷰, CHECK 제약 조건, DEFAULT 정의

개체에 대해 따옴표 붙은 식별자 설정을 ON으로 지정합니다. 이는 개체 정의에 관련된 모든 식에서 큰따옴표가 식별자를 구분함을 의미합니다.

1 = ON

0 = OFF

IsQueue

임의의 스키마 범위 개체

Service Broker 큐

1 = True

0 = False

IsReplProc

임의의 스키마 범위 개체

복제 프로시저입니다.

1 = True

0 = False

IsRule

임의의 스키마 범위 개체

바운드 규칙입니다.

1 = True

0 = False

IsScalarFunction

함수

스칼라 반환 함수입니다.

1 = 스칼라 반환 함수

0 = 스칼라 반환 함수 아님

IsSchemaBound

함수, 뷰

SCHEMABINDING을 사용하여 만든 스키마 바운드 함수 또는 뷰입니다.

1 = 스키마 바운드

0 = 스키마 바운드가 아닙니다.

IsSystemTable

테이블

시스템 테이블입니다.

1 = True

0 = False

IsTable

테이블

테이블입니다.

1 = True

0 = False

IsTableFunction

함수

테이블 반환 함수입니다.

1 = 테이블 반환 함수

0 = 테이블 반환 함수 아님

IsTrigger

임의의 스키마 범위 개체

트리거입니다.

1 = True

0 = False

IsUniqueCnst

임의의 스키마 범위 개체

UNIQUE 제약 조건입니다.

1 = True

0 = False

IsUserTable

테이블

사용자 정의 테이블입니다.

1 = True

0 = False

IsView

뷰입니다.

1 = True

0 = False

OwnerId

임의의 스키마 범위 개체

개체의 소유자입니다.

참고참고
스키마 소유자가 반드시 개체 소유자일 필요는 없습니다. 예를 들어 자식 개체(parent_object_id가 Null이 아닌 개체)는 항상 부모 개체와 같은 소유자 ID를 반환합니다.

Null이 아닌 경우 = 개체 소유자의 데이터베이스 사용자 ID입니다.

TableDeleteTrigger

테이블

테이블에 DELETE 트리거가 있습니다.

>1 = 지정된 유형의 첫 번째 트리거 ID

TableDeleteTriggerCount

테이블

테이블에 지정된 개수의 DELETE 트리거가 있습니다.

>0 = DELETE 트리거의 수입니다.

TableFullTextMergeStatus

테이블

테이블에 현재 병합 중인 전체 텍스트 인덱스가 있는지를 나타냅니다.

0 = 테이블에 전체 텍스트 인덱스가 없거나 병합 중인 전체 텍스트 인덱스가 없습니다.

1 = 전체 텍스트 인덱스가 병합 중입니다.

TableFullTextBackgroundUpdateIndexOn

테이블

테이블이 활성화된 전체 텍스트 백그라운드 업데이트 인덱스(자동 변경 추적)를 가집니다.

1 = TRUE

0 = FALSE

TableFulltextCatalogId

테이블

테이블의 전체 텍스트 인덱스 데이터가 있는 전체 텍스트 카탈로그 ID입니다.

0이 아닌 값 = 전체 텍스트 인덱싱된 테이블의 행을 식별하는 고유 인덱스와 연결된 전체 텍스트 카탈로그 ID입니다.

0 = 테이블에 전체 텍스트 인덱스가 없습니다.

TableFulltextChangeTrackingOn

테이블

테이블의 전체 텍스트 변경 내용 추적이 활성화되었습니다.

1 = TRUE

0 = FALSE

TableFulltextDocsProcessed

테이블

전체 텍스트 인덱싱이 시작된 이후에 처리된 행의 수입니다. 전체 텍스트 검색을 위해 인덱싱 중인 테이블에서 한 행의 모든 열은 인덱싱되는 문서의 일부로 간주됩니다.

0 = 활성 탐색 또는 전체 텍스트 인덱싱이 완료되지 않았습니다.

> 0 = 다음 중 하나일 수 있습니다.

  • 전체, 증분 또는 수동 변경 내용 추적 채우기가 시작된 이후에 삽입 또는 업데이트 작업으로 처리된 문서의 수입니다.

  • 백그라운드 업데이트 인덱스 채우기가 지정된 변경 내용 추적이 활성화되거나 전체 텍스트 인덱스 스키마가 변경되거나 전체 텍스트 카탈로그가 다시 작성되거나 SQL Server 인스턴스가 다시 시작되는 등의 작업이 수행된 이후 삽입 또는 업데이트 작업에 의해 처리된 행의 수입니다.

NULL = 테이블에 전체 텍스트 인덱스가 없습니다.

참고참고
이 속성은 삭제된 행을 모니터링하거나 세지 않습니다.

TableFulltextFailCount

테이블

전체 텍스트 검색이 인덱싱하지 않은 행의 수입니다.

0 = 채우기가 완료되었습니다.

> 0 = 다음 중 하나일 수 있습니다.

  • 전체, 증분 및 수동 업데이트 변경 내용 추적 채우기가 시작된 이후에 인덱싱되지 않은 문서의 수입니다.

  • 백그라운드 업데이트 인덱스가 지정된 변경 내용 추적에서 채우기가 시작 또는 다시 시작된 후에 인덱싱되지 않은 행의 수입니다. 그 원인은 스키마 변경, 카탈로그 다시 작성, 서버 다시 시작 등이 될 수 있습니다.

NULL = 테이블에 전체 텍스트 인덱스가 없습니다.

TableFulltextItemCount

테이블

성공적으로 전체 텍스트 인덱싱된 행의 수입니다.

TableFulltextKeyColumn

테이블

전체 텍스트 인덱스 정의에 사용되는 단일 열의 고유 인덱스와 관련된 열의 ID입니다.

0 = 테이블에 전체 텍스트 인덱스가 없습니다.

TableFulltextPendingChanges

테이블

처리할 보류 중인 변경 내용 추적 항목의 수입니다.

0 = 변경 내용 추적이 활성화되지 않았습니다.

NULL = 테이블에 전체 텍스트 인덱스가 없습니다.

TableFulltextPopulateStatus

테이블

0 = 유휴 상태

1 = 전체 채우기가 진행 중입니다.

2 = 증분 채우기가 진행 중입니다.

3 = 추적된 변경 내용의 전파가 진행 중입니다.

4 = 변경 내용 자동 추적과 같은 백그라운드 업데이트 인덱스가 진행 중입니다.

5 = 전체 텍스트 인덱싱이 정체 또는 일시 중지되었습니다.

TableHasActiveFulltextIndex

테이블

테이블에 활성 전체 텍스트 인덱스가 있습니다.

1 = True

0 = False

TableHasCheckCnst

테이블

테이블에 CHECK 제약 조건이 있습니다.

1 = True

0 = False

TableHasClustIndex

테이블

테이블에 클러스터형 인덱스가 있습니다.

1 = True

0 = False

TableHasDefaultCnst

테이블

테이블에 DEFAULT 제약 조건이 있습니다.

1 = True

0 = False

TableHasDeleteTrigger

테이블

테이블에 DELETE 트리거가 있습니다.

1 = True

0 = False

TableHasForeignKey

테이블

테이블에 FOREIGN KEY 제약 조건이 있습니다.

1 = True

0 = False

TableHasForeignRef

테이블

테이블이 FOREIGN KEY 제약 조건에 의해 참조됩니다.

1 = True

0 = False

TableHasIdentity

테이블

테이블에 ID 열이 있습니다.

1 = True

0 = False

TableHasIndex

테이블

테이블에 임의 유형의 인덱스가 있습니다.

1 = True

0 = False

TableHasInsertTrigger

테이블

개체에 INSERT 트리거가 있습니다.

1 = True

0 = False

TableHasNonclustIndex

테이블

테이블에 비클러스터형 인덱스가 있습니다.

1 = True

0 = False

TableHasPrimaryKey

테이블

테이블에 기본 키가 있습니다.

1 = True

0 = False

TableHasRowGuidCol

테이블

테이블에 uniqueidentifier 열에 대한 ROWGUIDCOL이 있습니다.

1 = True

0 = False

TableHasTextImage

테이블

테이블에 textntext 또는 image 열이 있습니다.

1 = True

0 = False

TableHasTimestamp

테이블

테이블에 timestamp 열이 있습니다.

1 = True

0 = False

TableHasUniqueCnst

테이블

테이블에 UNIQUE 제약 조건이 있습니다.

1 = True

0 = False

TableHasUpdateTrigger

테이블

개체에 UPDATE 트리거가 있습니다.

1 = True

0 = False

TableHasVarDecimalStorageFormat

테이블

테이블을 vardecimal 저장소 형식에 사용할 수 있습니다.

1 = True

0 = False

TableInsertTrigger

테이블

테이블에 INSERT 트리거가 있습니다.

>1 = 지정된 유형의 첫 번째 트리거 ID

TableInsertTriggerCount

테이블

테이블에 지정된 개수의 INSERT 트리거가 있습니다.

>0 = INSERT 트리거의 수입니다.

TableIsFake

테이블

실제 테이블이 아닙니다. SQL Server 데이터베이스 엔진의 요청이 있을 때 내부적으로 구체화됩니다.

1 = True

0 = False

TableIsLockedOnBulkLoad

테이블

bcp 또는 BULK INSERT 작업으로 인해 테이블이 잠깁니다.

1 = True

0 = False

TableIsPinned

테이블

테이블이 데이터 캐시에 보유되도록 고정됩니다.

0 = False

SQL Server 2005 이상에서는 이 기능이 지원되지 않습니다.

TableTextInRowLimit

테이블

text in row에 최대 바이트가 허용됩니다.

text in row 옵션을 설정하지 않으면 0입니다.

TableUpdateTrigger

테이블

테이블에 UPDATE 트리거가 있습니다.

>1 = 지정된 유형의 첫 번째 트리거 ID

TableUpdateTriggerCount

테이블

테이블에 지정된 개수의 UPDATE 트리거가 있습니다.

> 0 = UPDATE 트리거의 수입니다.

TableHasColumnSet

테이블

테이블에 열 집합이 있습니다.

0 = False

1 = True

자세한 내용은 열 집합 사용을 참조하십시오.

오류가 발생하거나 호출자가 개체를 볼 수 있는 권한을 갖고 있지 않으면 NULL을 반환합니다.

사용자는 소유하고 있거나 사용 권한을 부여 받은 보안 개체의 메타데이터만 볼 수 있습니다. 즉, 사용자가 개체에 대한 사용 권한이 없으면 OBJECTPROPERTY와 같은 메타데이터 내보내기 기본 제공 함수가 NULL을 반환합니다. 자세한 내용은 메타데이터 표시 유형 구성 및 메타데이터 표시 문제 해결을 참조하십시오.

데이터베이스 엔진에서는 object_id가 현재 데이터베이스 컨텍스트에 있다고 가정합니다. 다른 데이터베이스의 object_id를 참조하는 쿼리는 NULL 또는 잘못된 결과를 반환합니다. 예를 들어 다음 쿼리에서 현재 데이터베이스 컨텍스트는 master 데이터베이스입니다. 데이터베이스 엔진은 쿼리에서 지정된 데이터베이스 대신 현재 데이터베이스에서 지정된 object_id에 대한 속성 값을 반환하려고 합니다. master 데이터베이스에는 vEmployee 뷰가 없으므로 이 쿼리는 잘못된 결과를 반환합니다.

USE master;
GO
SELECT OBJECTPROPERTY(OBJECT_ID(N'AdventureWorks2008R2.HumanResources.vEmployee'), 'IsView');
GO

OBJECTPROPERTY(view_id, 'IsIndexable')는 IsIndexable 속성 평가에 뷰 정의의 구문 분석, 정규화 및 부분 최적화를 요구하므로 많은 컴퓨터 리소스를 사용할 수 있습니다. IsIndexable 속성은 인덱싱 가능한 테이블이나 뷰를 식별할 수 있지만 인덱스 키에 대한 특정한 요구가 만족되지 않으면 인덱스가 실제로 생성되지 않을 수 있습니다. 자세한 내용은 CREATE INDEX(Transact-SQL)를 참조하십시오.

OBJECTPROPERTY(table_id, 'TableHasActiveFulltextIndex')는 인덱싱을 위해 하나 이상의 테이블 열을 추가한 경우 값 1(true)을 반환합니다. 전체 텍스트 인덱싱은 첫 번째 열이 인덱싱용으로 추가되자마자 채우기용으로 활성화됩니다.

테이블이 생성될 때 QUOTED IDENTIFIER 옵션이 OFF로 설정된 경우에도 해당 테이블의 메타데이터에는 항상 ON으로 저장됩니다. 따라서 OBJECTPROPERTY(table_id, 'IsQuotedIdentOn')는 항상 값 1(true)을 반환합니다.

1. 개체가 테이블인지 확인

다음 예에서는 UnitMeasure가 AdventureWorks2008R2 데이터베이스에 있는 테이블인지 테스트하는 방법을 보여 줍니다.

USE AdventureWorks2008R2;
GO
IF OBJECTPROPERTY (OBJECT_ID(N'Production.UnitMeasure'),'ISTABLE') = 1
   PRINT 'UnitMeasure is a table.'
ELSE IF OBJECTPROPERTY (OBJECT_ID(N'Production.UnitMeasure'),'ISTABLE') = 0
   PRINT 'UnitMeasure is not a table.'
ELSE IF OBJECTPROPERTY (OBJECT_ID(N'Production.UnitMeasure'),'ISTABLE') IS NULL
   PRINT 'ERROR: UnitMeasure is not a valid object.';
GO


2. 스칼라 반환 사용자 정의 함수가 결정적인지 확인

다음 예에서는 money 값을 반환하는 사용자 정의 스칼라 반환 함수인 ufnGetProductDealerPrice가 결정적인지 여부를 테스트하는 방법을 보여 줍니다.

USE AdventureWorks2008R2;
GO
SELECT OBJECTPROPERTY(OBJECT_ID('dbo.ufnGetProductDealerPrice'), 'IsDeterministic');
GO


결과 집합은 ufnGetProductDealerPrice가 결정적 함수가 아님을 보여 줍니다.

-----

0

3. 특정 스키마에 속하는 개체 찾기

다음 예에서는 SchemaId 속성을 사용하여 Production 스키마에 속하는 모든 개체를 반환합니다.

USE AdventureWorks2008R2;
GO
SELECT name, object_id, type_desc
FROM sys.objects 
WHERE OBJECTPROPERTY(object_id, N'SchemaId') = SCHEMA_ID(N'Production')
ORDER BY type_desc, name;
GO


posted by LifeisSimple
2011. 5. 6. 17:28 Brain Trainning/DataBase
MSSQL -> MySQL 링크드 관련해서 잘 정리되어 있습니다. 

출처 : http://www.packtpub.com/article/mysql-linked-server-on-sql-server-2008

by Jayaram Krishnaswamy | August 2009 | .NET Microsoft

Linking servers provides an elegant solution when you are faced with running queries against databases on distributed servers or looking at your distributed assets on disparate databases.

This article by Dr. Jay Krishnaswamy explains how to set up a MySQL linked server on SQL Server 2008 Enterprise. Configuring a linked MySQL server as well as querying a table on the MySQL linked server is described. The reader would benefit reviewing the first article on this series on MySQL Servers.

Introduction

MS SQL servers always provided remote access to servers through RPC mechanisms, but they had the limitation of being confined to invoking procedures on remote locations. A linked server (a virtual server) may be considered a more flexible way of achieving the same thing, with the added benefits of remote table access and distributed queries. Microsoft manages the link mechanism via OLE DB technology. Specifically, an OLE DB datasource points to the specific database that can be accessed using OLEDB.

In this article we will be creating a MySQL linked server on SQL Server 2008 and querying a database [TestMove] table shown in the next listing . In reviewing the previous article it may be noticed that the Employees tables was moved to MySQL database TestMove. In running the commands from the mysql>prompt it is assumed that the MySQL Server has been started.

Listing 1: employees table in TestMove

mysql> show tables;
+--------------------+
| Tables_in_testmove |
+--------------------+
| employees |
+--------------------+
1 row in set (0.09 sec)

mysql>

Creating an ODBC DSN for MySQL

In the previous article on MySQL Servers cited earlier, a DSN was created for moving data. Essentially the same DSN can be used. Herein follows a brief review of the DSN MySQL_Link created along the same lines as in the previous referenced article. The ODBC driver used for creating this ODBC DSN is the one installed on the machine when the MySQL Server was installed as shown.

MySQL Linked Server on SQL Server 2008

The final interactive window where you may test the connectivity is shown in the next figure. You may notice that the database Testmove has been named in the ODBC DSN. The name MySQL_LINK is the ODBC DSN.

MySQL Linked Server on SQL Server 2008

When you close the window after clicking the OK button, a ODBC DSN item will be added to the System DSN tab of the ODBC wizard as shown.

MySQL Linked Server on SQL Server 2008

Steps to create a linked server from Management Studio

Right click the Linked Servers node to display a drop-down menu as shown in the next figure.

MySQL Linked Server on SQL Server 2008

Click on New Linked Server...item. This brings up the New Linked Server window as shown. The window is all empty except for a default Provider.

MySQL Linked Server on SQL Server 2008

The very first thing to do is to provide a name for this linked server. Herein it is LINKED_ MYSQL. We will be using one of the providers [Microsoft OLE DB Provider for ODBC] supported by SQL Server 2008. You may access the list of OLE DB Providers in the Providers sub-folder of the Linked Servers. Provide the other details as shown in the next figure. Make sure they are entered exactly as shown or according to how you have created the database on MySQL Server.

MySQL Linked Server on SQL Server 2008

Click on the Security list item under General in the left. This opens the 'Security' page of the New Linked Server wizard as shown. Change the login from the default "Be made without using a security context" to "Be made using this security context". Enter remote login. In this case it is "root" for the remote login and the password is the one used during the ODBC DSN (also the password for server authentication) creation.

MySQL Linked Server on SQL Server 2008

Make no changes to the Server Options page. Click OK. This creates a linked server Linked_MySQL as shown expanded in the Linked Server's node as shown. You may need to right click and refresh the Linked Servers' node to see the new linked server. As you can see in the figure, the 'User' tables are not displayed.

MySQL Linked Server on SQL Server 2008

 

 

Running Queries and reviewing results

Running system stored procedures can provide various levels of information and the database can be queried using the four part syntax and the openquery() method.

Information on the linked server

It is easy to find how the linked server is configured using system stored procedure sp_linkedsrvlogin on the SQL Server 2008. Open a Query window from File | New | Query Current Connection to open the query window and type in the following statement. The next figure shows the statement as well as the returned values. SQL Server 2008 querying has the intellisense report and this must be put to good use.

Exec sp_linkedsrvlogin

This shows all servers both local and remote as shown in the next figure.

MySQL Linked Server on SQL Server 2008

Information about the tables on the remote server can also be accessed by running a stored procedure. Executing the stored procedure sp_tables_ex as shown in the next figure (which displays the statement and the result of executing the stored procedure) can be used to obtain table information.

MySQL Linked Server on SQL Server 2008

Querying the table on the database

Data in the table on the linked server can be queried using the openquery() function. The syntax for this function shown next is very simple.

openquery ('linked server', 'query')

The next figure shows the result of running the openquery() function on the Linked_MySQL linked server.

MySQL Linked Server on SQL Server 2008

Although it should be possible to query the linked server using the four part syntax as in:

Select * from LINKED_MYSQL...employees

The above statement returns an error. This is probably a limitation of a combination of MSDASQL and the ODBC driver which does not provide the schema information correctly(this is just the author's opinion).

Are Remote Procedure Calls (RPC) allowed?

The easiest way to test this is to send out a call by running the following query against the linked server.

Execute('Select FirstName, LastName from employees') at Linked_MYSQL

If the linked server is not configured for RPC, then the result you get by running the above query is as shown in the next figure.

MySQL Linked Server on SQL Server 2008

Turn on RPC

Earlier on we skipped the Server Options page of the linked server. Back in the Management Studio right click linked server LINKED_MYSQL and from the drop-down choose to look at the properties at the bottom of the list. This brings up the LINKED_MYSQL properties window as shown. Click on Server Options. In the Server Options page change the values of RPC and RPCOUT to true, default for both being false.

MySQL Linked Server on SQL Server 2008

Now run the query that produced the error previously. The result is displayed in the next figure.

MySQL Linked Server on SQL Server 2008

You might have noted that only two columns were returned from the employees table. This was deliberate as trying to get all the column would produce an error due partly to the data types of data stored in the table and their compatibility with MSDASQL and the ODBC driver (Again, an author's opinion).

Creating Linked Server using TSQL

While the linked server can be created using the built-in wizard of the Management Studio, it can also be created using TSQL statements as in the following listing (run both statements, the first one creates the linked server and the second the logins).

Listing 2:

 Exec master.dbo.sp_addlinkedserver
@server=N'MySQlbyCode',
@srvprodcut=N'MySQL',
@provider=N'MSDASQL',
@datasrc=N'MySQL_link'

Exec master.dbo.sp_addlinkedserverlogin
@server=N'MySQlbyCode',
@locallogin=NULL,
@rmtuser=N'root',
@rmtpassword=N'<your password>'
@rmtsrvname=N'localhost'

Summary

The article described the steps involved in configuring a MySql Linked server on SQL Server 2008 using the built-in New Linked Server wizard as well as TSQL. Method to query the linked server as well as enabling RPC were described.

posted by LifeisSimple
2011. 4. 28. 18:04 Brain Trainning/DataBase
출처 : http://technet.microsoft.com/en-us/library/cc280449.aspx

Creating Compressed Tables and Indexes

SQL Server 2008 supports both row and page compression for both tables and indexes. Data compression can be configured for the following database objects:

  • A whole table that is stored as a heap.

  • A whole table that is stored as a clustered index.

  • A whole nonclustered index.

  • A whole indexed view.

  • For partitioned tables and indexes, the compression option can be configured for each partition, and the various partitions of an object do not have to have the same compression setting.

The compression setting of a table is not automatically applied to its nonclustered indexes. Each index must be set individually. Compression is not available for system tables. Tables and indexes can be compressed when they are created by using theCREATE TABLE and CREATE INDEX statements. To change the compression state of a table, index, or partition, use the ALTER TABLE or ALTER INDEX statements.

NoteNote

If the existing data is fragmented, you might be able to reduce the size of the index by rebuilding the index instead of using compression. The fill factor of an index will be applied during an index rebuild, which could potentially increase the size of the index. For more information, see Fill Factor.

When you use row and page compression, be aware the following considerations:

  • Compression is available only in the SQL Server 2008 Enterprise and Developer editions.

  • Compression can allow more rows to be stored on a page, but does not change the maximum row size of a table or index.

  • A table cannot be enabled for compression when the maximum row size plus the compression overhead exceeds the maximum row size of 8060 bytes. For example, a table that has the columns c1 char(8000) and c2 char(53) cannot be compressed because of the additional compression overhead. When the vardecimal storage format is used, the row-size check is performed when the format is enabled. For row and page compression, the row-size check is performed when the object is initially compressed, and then checked as each row is inserted or modified. Compression enforces the following two rules:

    • An update to a fixed-length type must always succeed.

    • Disabling data compression must always succeed. Even if the compressed row fits on the page, which means that it is less than 8060 bytes; SQL Server prevents updates that would not fit on the row when it is uncompressed.

  • When a list of partitions is specified, the compression type can be set to ROW, PAGE, or NONE on individual partitions. If the list of partitions is not specified, all partitions are set with the data compression property that is specified in the statement. When a table or index is created, data compression is set to NONE unless otherwise specified. When a table is modified, the existing compression is preserved unless otherwise specified.

  • If you specify a list of partitions or a partition that is out of range, an error will be generated.

  • Nonclustered indexes do not inherit the compression property of the table. To compress indexes, you must explicitly set the compression property of the indexes. By default, the compression setting for indexes will set to NONE when the index is created.

  • When a clustered index is created on a heap, the clustered index inherits the compression state of the heap unless an alternative compression state is specified.

  • When a heap is configured for page-level compression, pages receive page-level compression only in the following ways:

    • Data is inserted by using the BULK INSERT syntax.

    • Data is inserted using INSERT INTO ... WITH (TABLOCK) syntax.

    • A table is rebuilt by executing the ALTER TABLE ... REBUILD statement with the PAGE compression option.

  • New pages allocated in a heap as part of DML operations will not use PAGE compression until the heap is rebuilt. Rebuild the heap by removing and reapplying compression, or by creating and removing a clustered index.

  • Changing the compression setting of a heap requires all nonclustered indexes on the table to be rebuilt so that they have pointers to the new row locations in the heap.

  • You can enable or disable ROW or PAGE compression online or offline. Enabling compression on a heap is single threaded for an online operation.

  • The disk space requirements for enabling or disabling row or page compression are the same as for creating or rebuilding an index. For partitioned data, you can reduce the space that is required by enabling or disabling compression for one partition at a time.

  • To determine the compression state of partitions in a partitioned table, query the data_compression column of the sys.partitions catalog view.

  • When you are compressing indexes, leaf-level pages can be compressed with both row and page compression. Non–leaf-level pages do not receive page compression.

  • Because of their size, large-value data types are sometimes stored separately from the normal row data on special purpose pages. Data compression is not available for the data that is stored separately.

  • Tables which implemented the vardecimal storage format in SQL Server 2005 will retain that setting when upgraded. You can apply row compression to a table that has the vardecimal storage format. However, because row compression is a superset of the vardecimal storage format, there is no reason to retain the vardecimal storage format. Decimal values gain no additional compression when you combine the vardecimal storage format with row compression. You can apply page compression to a table that has the vardecimal storage format; however, the vardecimal storage format columns probably will not achieve additional compression.

    NoteNote

    SQL Server 2008 supports the vardecimal storage format; however, because row-level compression achieves the same goals, the vardecimal storage format is deprecated. This feature will be removed in a future version of Microsoft SQL Server. Avoid using this feature in new development work, and plan to modify applications that currently use this feature.

For a summary of the implementation of data compression, see Row Compression ImplementationPage Compression Implementation, and Unicode Compression Implementation.

To determine how changing the compression state will affect a table or an index, use thesp_estimate_data_compression_savings stored procedure. The sp_estimate_data_compression_savings stored procedure is available only in the editions of SQL Server that support data compression.

When you are using data compression with partitioned tables and indexes, be aware of the following considerations:

  • Splitting a range

    When partitions are split by using the ALTER PARTITION statement, both partitions inherit the data compression attribute of the original partition.

  • Merging a range

    When two partitions are merged, the resultant partition inherits the data compression attribute of the destination partition.

  • Switching partitions

    To switch a partition, the data compression property of the partition must match the compression property of the table.

  • Rebuilding one partition or all partitions

    There are two syntax variations that you can use to modify the compression of a partitioned table or index:

    • The following syntax rebuilds only the referenced partition:

      ALTER TABLE <table_name> 
      REBUILD PARTITION = 1 WITH (DATA_COMPRESSION =  <option>)
    • The following syntax rebuilds the whole table by using the existing compression setting for any partitions that are not referenced:

      ALTER TABLE <table_name> 
      REBUILD PARTITION = ALL 
      WITH (DATA_COMPRESSION = PAGE ON PARTITIONS(<range>),
      ... )

    Partitioned indexes follow the same principle using ALTER INDEX.

  • Dropping a partitioned clustered index

    When a clustered index is dropped, the corresponding heap partitions retain their data compression setting unless the partitioning scheme is modified. If the partitioning scheme is changed, all partitions are rebuilt to an uncompressed state. To drop a clustered index and change the partitioning scheme requires the following steps:

    1. Drop the clustered index.

    2. Modify the table by using the ALTER TABLE ... REBUILD ... option that specifies the compression option.

    To drop a clustered index OFFLINE is a very fast operation, because only the upper levels of clustered indexes are removed. When a clustered index is dropped ONLINE, SQL Server must rebuild the heap two times, once for step 1 and once for step 2.

When you are using data compression with replication, be aware of the following considerations:

  • When the Snapshot Agent generates the initial schema script, the new schema will use the same compression settings for both the table and its indexes. Compression cannot be enabled on just the table and not the index.

  • For transactional replication the article schema option determines what dependent objects and properties have to be scripted. For more information, see sp_addarticle.

    The Distribution Agent does not check for down-level Subscribers when it applies scripts. If the replication of compression is selected, creating the table on down-level Subscribers will fail. In the case of a mixed topology, do not enable the replication of compression.

  • For merge replication, publication compatibility level overrides the schema options and determines the schema objects that will be scripted. For more information about compatibility level, see Using Multiple Versions of SQL Server in a Replication Topology.

    In the case of a mixed topology, if it is not required to support the new compression options, the publication compatibility level should be set to the down-level Subscriber version. If it is required, compress tables on the Subscriber after they have been created.

The following table shows replication settings that control compression during replication.

User intent

Replicate partition scheme for a table or index

Replicate compression settings

Scripting behavior

To replicate the partition scheme and enable compression on the Subscriber on the partition.

True

True

Scripts both the partition scheme and the compression settings.

To replicate the partition scheme but not compress the data on the Subscriber.

True

False

Scripts out the partition scheme but not the compression settings for the partition.

To not replicate the partition scheme and not compress the data on the Subscriber.

False

False

Does not script partition or compression settings.

To compress the table on the Subscriber if all the partitions are compressed on the Publisher, but not replicate the partition scheme.

False

True

Checks if all the partitions are enabled for compression.

Scripts out compression at the table level.

Compression occurs in the storage engine and the data is presented to most of the other components of SQL Server in an uncompressed state. This limits the effects of compression on the other components to the following:

  • Bulk import and export operations

    When data is exported, even in native format, the data is output in the uncompressed row format. This can cause the size of exported data file to be significantly larger than the source data.

    When data is imported, if the target table has been enabled for compression, the data is converted by the storage engine into compressed row format. This can cause increased CPU usage compared to when data is imported into an uncompressed table.

    When data is bulk imported into a heap with page compression, the bulk import operation will try to compress the data with page compression when the data is inserted.

  • Compression does not affect backup and restore.

  • Compression does not affect log shipping.

  • Enabling compression can cause query plans to change because the data is stored using a different number of pages and number of rows per page.

  • Data compression is supported by SQL Server Management Studio through the Data Compression Wizard.

To start the Data Compression Wizard

  • In Object Explorer, right-click a table, index, or indexed view, point to Storage, and then click Compress.

To monitor compression of the whole instance of SQL Server, use the Page compression attempts/sec and Pages compressed/sec counters of the SQL Server, Access Methods Object.

To obtain page compression statistics for individual partitions, query the sys.dm_db_index_operational_stats dynamic management function.

Some of the following examples use partitioned tables and require a database that has filegroups. To create a database that has filegroups, execute the following statement.

CREATE DATABASE TestDatabase
ON  PRIMARY
( NAME = TestDatabase,
    FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Data\TestDB.mdf'),
FILEGROUP test1fg
( NAME = TestDBFile1,
    FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Data\TestDBFile1.mdf'),
FILEGROUP test2fg
( NAME = TestDBFile2,
    FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Data\TestDBFile2.ndf'),
FILEGROUP test3fg
( NAME = TestDBFile3,
    FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Data\TestDBFile3.ndf'),
FILEGROUP test4fg
( NAME = TestDBFile4,
    FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Data\TestDBFile4.ndf') ;
GO

To switch to the new database:

USE TestDatabase
GO

A. Creating a table that uses row compression

The following example creates a table and sets the compression to ROW.

CREATE TABLE T1 
(c1 int, c2 nvarchar(50) )
WITH (DATA_COMPRESSION = ROW);
GO

B. Creating a table that uses page compression

The following example create a table and sets the compression to PAGE.

CREATE TABLE T2 
(c1 int, c2 nvarchar(50) )
WITH (DATA_COMPRESSION = PAGE);
GO

C. Setting the DATA_COMPRESSION option on a partitioned table

The following example uses the TestDatabase table that is created by using the code provided earlier in this section. The example creates a partition function and scheme, and then creates a partitioned table and specifies the compression options for the partitions of the table. In this example, partition 1 is configured for ROW compression, and the remaining partitions are configured for PAGE compression.

To create a partition function:

CREATE PARTITION FUNCTION myRangePF1 (int)
AS RANGE LEFT FOR VALUES (1, 100, 1000) ;
GO

To create a partition scheme:

CREATE PARTITION SCHEME myRangePS1
AS PARTITION myRangePF1
TO (test1fg, test2fg, test3fg, test4fg) ;
GO

To create a partitioned table that has compressed partitions:

CREATE TABLE PartitionTable1 
(col1 int, col2 varchar(max))
ON myRangePS1 (col1) 
WITH 
(
  DATA_COMPRESSION = ROW ON PARTITIONS (1),
  DATA_COMPRESSION = PAGE ON PARTITIONS (2 TO 4)
);
GO

D. Setting the DATA_COMPRESSION option on a partitioned table

The following example uses the database that is used in example C. The example creates a table by using the syntax for noncontiguous partitions.

CREATE TABLE PartitionTable2 
(col1 int, col2 varchar(max))
ON myRangePS1 (col1) 
WITH 
(
  DATA_COMPRESSION = ROW ON PARTITIONS (1,3),
  DATA_COMPRESSION = NONE ON PARTITIONS (2,4)
);
GO

E. Modifying a table to change the compression

The following example changes the compression of the nonpartitioned table that is created in example A.

ALTER TABLE T1 
REBUILD WITH (DATA_COMPRESSION = PAGE);
GO

F. Modifying the compression of one partition in a partitioned table

The following example changes the compression of the partitioned table that is created in example C. The REBUILD PARTITION = 1syntax causes only partition number 1 to be rebuilt.

ALTER TABLE PartitionTable1 
REBUILD PARTITION = 1 WITH (DATA_COMPRESSION =  NONE) ;
GO

The same operation that uses the following alternate syntax causes all partitions in the table to be rebuilt.

ALTER TABLE PartitionTable1 
REBUILD PARTITION = ALL 
WITH (DATA_COMPRESSION = PAGE ON PARTITIONS(1) ) ;
GO

G. Modifying the compression of a several partitions in a partitioned table

The REBUILD PARTITION = ... syntax can rebuild only one partition. To rebuild more than one partition, you must execute multiple statements, or execute the following example to rebuild all partitions, using the current compression settings for unspecified partitions.

ALTER TABLE PartitionTable1 
REBUILD PARTITION = ALL 
WITH
(
DATA_COMPRESSION = PAGE ON PARTITIONS(1), 
DATA_COMPRESSION = ROW ON PARTITIONS(2 TO 4) 
) ;
GO

H. Modifying the compression on an index

The following example uses the table that is created in example A and creates an index on the column C2.

CREATE NONCLUSTERED INDEX IX_INDEX_1 
    ON T1 (C2) 
WITH ( DATA_COMPRESSION = ROW ) ; 
GO

Execute the following code to change the index to page compression:

ALTER INDEX IX_INDEX_1 
ON T1
REBUILD WITH ( DATA_COMPRESSION = PAGE ) ;
GO

I. Modifying the compression of a single partition in a partitioned index

The following example creates an index on a partitioned table that uses row compression on all partitions of the index.

CREATE CLUSTERED INDEX IX_PartTab2Col1
ON PartitionTable1 (Col1)
WITH ( DATA_COMPRESSION = ROW ) ;
GO

To create the index so that is uses different compression settings for different partitions, use the ON PARTITIONS syntax. The following example creates an index on a partitioned table that uses row compression on partition 1 of the index and page compression on partitions 2 through 4 of the index.

CREATE CLUSTERED INDEX IX_PartTab2Col1
ON PartitionTable1 (Col1)
WITH (DATA_COMPRESSION = ROW ON PARTITIONS(1),
    DATA_COMPRESSION = PAGE ON PARTITIONS (2 TO 4 ) ) ;
GO

The following example changes the compression of the partitioned index.

ALTER INDEX IX_PartTab2Col1 ON PartitionTable1
REBUILD PARTITION = ALL 
WITH ( DATA_COMPRESSION = PAGE ON PARTITIONS(1) ) ;
GO

J. Modifying the compression of a several partitions in a partitioned index

The REBUILD PARTITION = ... syntax can rebuild only one partition. To rebuild more than one partition, you must execute multiple statements, or execute the following example to rebuild all partitions, using the current compression settings for unspecified partitions.

ALTER INDEX IX_PartTab2Col1 ON PartitionTable1
REBUILD PARTITION = ALL 
WITH
(
DATA_COMPRESSION = PAGE ON PARTITIONS(1), 
DATA_COMPRESSION = ROW ON PARTITIONS(2 TO 4) 
) ;
GO
posted by LifeisSimple
2011. 4. 22. 11:42 BookStory
산지 한달만에 읽기 시작한 책 
실전SQLSEVERMVP53
카테고리 컴퓨터/IT > 데이터베이스 > SQL서버 > SQL서버일반
지은이 MVP 커뮤니티 (비제이퍼블릭, 2010년)
상세보기

아마존에 원서가 있지만 그래도 번역자를 신뢰하여 구입한 책...

내용은 음... 와~ 하는 정도는 아니지만 그래도 기억의 저편에 혹은 당연하지만 잊고 지냈던것들을 잘 꺼내서 정리할 수 있도록 해주는 책.
한번은 정독할 가치가 있는 책이다... 그리고, 무엇보다 중요한건 직접테스트하고 실행해보는것...

이제 하나씩 읽으면서 정리를 하도록 하겠다 ㅎㅎ

데이터 무결성을 위한  SQL Server  도구들...

1. 데이터 형식 : 저장할 데이터에 맞는 기본 형식과 크기를 정의
2. NULL  설정 : 실제로 NULL 값이 발생할 수 있는 열에 대해서만 NULL 허용을 설정 (실제 개발자가 NULL 여부를 확인하는 조건절을 넣어도 알아서 무시하게 된다)
3. UNIQUE 제약 조건 : 열 또는 열 집합에 중복된 데이터가 발생하는 것을 방지
4. 필터링된 UNIQUE 인덱스 : 테이블에 있는 일련의 데이터를 구성하는 열 또는 열 집합에 중복된 데이터가 발생되는 것을 방지
5. 참조 키 제약 조건 : 모든 참조 키를 설정된 값들은 연관된 테이블에 기본 키로 정의된 값들 중 하나의 값만 허용
6. 체크 제약 조건 : 행에 있는 갓ㅂ들이 최소 기준을 만족하는지 검사 (너무 열심히 걸어주면 Insert 에 오버헤드가 발생할 수 있다.)
7. 트리거 : 이를 통해 모든 제약 조건들을 구현할 수 있음. DML 명령이 수행될 때마다 설정해놓은 배치 코드를 실행. 이 외에도 사용자가 알지 못하도록 하면서 부수적인 동작을 수행하도록 한다. (그러나, 개발자가 기억하지 못하면 이후에 원인분석으로 인한 고생을 할 수 있다.) 


여러가지 자세한 내용은 책을 직접 읽어볼것....  
posted by LifeisSimple
2011. 4. 20. 11:53 Brain Trainning/DataBase
출처 : http://blogs.msdn.com/b/sqlcat/archive/2007/06/20/distributed-partitioned-views-federated-databases-lessons-learned.aspx

Distributed Partitioned Views / Federated Databases: Lessons Learned

RATE THIS

Introduction

This article contains information about the things we have learned while working with Federated Databases.  Before beginning it is necessary to define the terms being used.  Included in this article is one solution in production that is using federated servers and Distributed Partitioned Views.

 

Definitions

Definition 1:  Local Partitioned View – A single table is horizontally split into multiple tables, usually all have the same structure.

 

Example 1:

Original table is named FACT.

New tables are named FACT2005, FACT2006, and FACT2007.  (I chose to partition by date, you can choose anything else that makes sense for your application, like branch office).

 

CREATE VIEW FACT AS

SELECT <select list> FROM dbo.FACT2005

UNION ALL

SELECT <select list> FROM dbo.FACT2006

UNION ALL

SELECT <select list> FROM dbo.FACT2007

 

You will notice that the view is created with the same name as the original table.  This way the applications don’t have to change.

 

You also have to create a check constraint on the same column in each table to create a partitioning “key”.  Read more about this and other very important restrictions in Books on Line at: ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/aecc2f73-2ab5-4db9-b1e6-2f9e3c601fb9.htm.  There are several good places in Books on Line to read about partitioned views; this link will give you a launching point for the other ones.

 

Note:  Local partitioned views are popular in both OLTP and data warehouse projects.  We rarely hear about problems with these, but there are known problems with very complex queries when the optimizer does not do partition elimination.  I will not spend much time on local partitioned views in this article as I would like to concentrate on Distributed Partitioned Views.

 

Definition 2: Cross Database Partitioned View – tables are split among different databases on the same server instance. Notice the three part name using the database in Example 2 below.

 

Example 2: 

CREATE VIEW FACT AS

SELECT <select list> FROM DB2005.dbo.FACT2005

UNION ALL

SELECT <select list> FROM DB2006.dbo.FACT2006

UNION ALL

SELECT <select list> FROM DB2007.dbo.FACT2007

 

Note:  The most frequent question I get from people attempting a view like this that has tables in multiple databases on the same instance is about joins.  You don’t lose too much performance with cross database joins.  This is something to think about because you will normally join this view to some reference tables for the application.  If you carry the reference tables in a database called COMMON for example, then you will most likely see something like SELECT * FROM dbo.FACT JOIN COMMON.dbo.Customer ON ….. WHERE …

 

You will also notice in this example that I put each fact table in its own database to facilitate easier scale out across servers or instances if you need to do this at a later time.

 

 

Definition 3: Distributed (across server or instance) Partitioned View.  Tables participating in the view reside in different databases which reside on different servers or different instances.  Note the four part name which includes the actual server name (or the cluster name if this is in a Windows Failover Cluster).

 

Example 3: 

CREATE VIEW FACT AS

SELECT <select list> FROM              DB2005.dbo.FACT2005

UNION ALL

SELECT <select list> FROM server2.DB2006.dbo.FACT2006

UNION ALL

SELECT <select list> FROM server3.DB2007.dbo.FACT2007

 

You will notice that the server name is missing from the first server.  This view definition in Example 3 exists on server1.  You cannot use a linked server to refer to the local server.    You might immediately recognize a potential problem and try to create the view in Example 4 on server 2.  However, the trick is to change the linked server definitions so that the same view code in Example 3 is deployed to every server. 

 

BAD Example 4:  (don’t do this)

CREATE VIEW FACT AS

SELECT <select list> FROM  server1.DB2005.dbo.FACT2005

UNION ALL

SELECT <select list> FROM               DB2006.dbo.FACT2006

UNION ALL

SELECT <select list> FROM server3.DB2007.dbo.FACT2007

 

Instead of doing the BAD Example 4, use the style in Example 3 and change the linked server definitions on every server.

 

 

Infrastructure architecture for one DPV project

Here is one production project to be used as a reference for DPV.  The customer had 3,000 branch offices to automate and determined that the total size representing 3 months worth of data was going to be approximately 3.6TB.  They didn’t want to risk putting this all on one server, so their basic approach was to break this into manageable size pieces.  There are 6 servers in 2 geographically separated data centers.  Each server has 2 instances so that if one server gets too busy they can easily move the second instance to another server.  The data is spread out over 12 instances.  The servers are 8 socket dual core machines with 16GB RAM.  Each instance is responsible for approximately 300GB.  Growth is expected to double soon which is why they selected machines this big.

 

Another reason they bought bigger machines than they need for a normal load is to have a high availability strategy.  The three machines in each data center are in one cluster and if one machine goes down another machine can pick up the load.  If two machines go down, one machine will do the work of all 3 and they expect that performance will be slow until the problem is fixed.  If an entire data center goes down there is no solution in place yet – this is a later phase of the project.

 

The most important point in the success of this project and makes this project work so well is that they are not using load balancing.  The users at each branch are connected directly to a server that contains their data.  So even though most of the inserts, updates and deletes are done through the partitioned views, the work is mostly local to one server.  There are some corporate users issuing queries that need data from multiple instances and it is expected that most of these queries will touch multiple servers.

 

Reasons for Distributing:

The main reason for distributing the 3.6TB over 12 servers was to break the data into manageable size pieces.  This makes the hard maintenance problem in a VLDB much easier to deal with; i.e. backups, index and statistics maintenance. 

 

If you use table partitioning as well as the partitioned views, then you have even smaller pieces to manage for most of your administrative processes.

 

 

WARNING:

Although some federated server projects are successful, we have found several problems during these projects.  Fortunately we found solutions for each problem.  The warning is that we don’t know if we have uncovered all the problems.  Your workload may be different enough that other problems will be discovered.  So test, test, test if you decide to distribute a single database over multiple servers.

 

 

Two main problem patterns

The two main problems we have found so far are:

1)      When a command gets sent to every server when you think it should only go to one server.  This happens when the query optimizer thinks it has to check the schema on every server, as in the case when the same collations are not used on all the servers (see note below in the lessons learned section).

2)      When cross-server join copies the records from the remote server to the local server and then performs the join.  This is called a non-remotable query.  The optimizer is pretty good at copying the smaller table (or result set) to the right server before performing the join.  Still, it is a situation that should be avoided in order to get the most consistent performance.  Try to make all the joins happen on one server (either all on the remote or all on the local) without copying records across the network.  See the notes below on advice on how to avoid this.

 

 

Here is a list of the things we have learned to do and not do on this project. 

 

Lessons Learned on Distributed Partitioned Views: (multiple servers involved)

  1. Follow the guidelines in Books On Line very carefully.  There are many links from this main one.  I recommend reading and re-reading these until you know the subject very well before you start.  ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/6e44b9c2-035e-4c88-907f-eef880c5540e.htm.  If you are on-line and can get to this site: http://msdn2.microsoft.com/en-us/library/ms175129(SQL.90).aspx, read it and any of the links it has on this page.
  2. Avoid cross server joins whenever possible.  One solution is to replicate all your reference/dimension tables to every server.  In a cross server join, the necessary records are copied from the remote server to the local server, then the join is performed. 
  3. Use the same collations in all databases.  Otherwise the startup filters are not applied and the queries are always sent to servers with different collation.
  4. Use the same session settings in all connections.  Otherwise startup filters are not applied and the queries are always sent to servers with different session settings.
  5. Create an index with the partitioned column as the leading column on the index because most of your queries will contain the partitioned column in the where clause.  The optimizer uses this index and the associated statistics to do more efficient queries.
  6. Plan guides do not work for distributed queries.
  7. After restarting SQL Server, create some startup stored procedures that will run the queries that you need.  Otherwise the first user running each query will pay a heavier penalty because it has to touch every server.  These startup procedures will also create a local connection pool to each server.  Even though creating a connection is fast it will still be better if the first user doesn’t have to wait for this too. 
  8. Don’t forget to do the tip in Books-On-Line:  Turn on Lazy Schema Validation.  This will give you better performance.  It helps to avoid sending all queries to all servers.
  9. Always have the correct data type in your statements.  Avoid situations where SQL Server has to automatically convert to a type in your table.  Otherwise the query always gets sent to every server for execution.   In the following example, the partition key column (name) is defined in all the tables as NVARCHAR.

 

DON’T DO THIS

SELECT column1 FROM myPartitionedView WHERE name = ‘ABC’  -- note ‘ABC’ is interpreted as VARCHAR.

 

DO THIS

SELECT column1 FROM myPartitionedView WHERE name = N’ABC’

Or

SELECT column1 FROM myPartitionedView WHERE name = CAST (NVARCHAR, ‘ABC’)

 

  1. Don’t use non-deterministic functions directly in your insert or update statements.   This causes the optimizer to send the query to all servers.

 

DON’T DO THIS

UPDATE myPartitionedView SET column1 = GETDATE() WHERE …

 

DO THIS

DECLARE @dtNow datetime

SET @dtNow = GETDATE()

UPDATE myPartitionedView SET column1 = @dtNow WHERE …

 

  1. The Estimated plan appears to show that all servers will be touched all the time.  But in reality the startup filter will eliminate the partitions at run time.  This is by design and the challenge is to teach yourself about the definition of a startup filter.  You can find a good write up at: http://www.sqlmag.com/articles/index.cfm?ArticleID=9097.  (you must have an account to read this).  I have not found much else written on the topic of a start up filter and I wish I could paste a screen capture in this blog because it is easier to understand once you see the picture.
  2. To avoid the problem of some high severity errors not getting returned properly from the remote server use the following line of code:

DO THIS

SET IMPLICIT_TRANSACTIONS ON

 

What happens is that the Try…Catch blocks are not activated properly on the local server.  For example, if the following insert generated a duplicate key error, then the whole batch aborts and the Commit Transaction as well as the Try…Catch are ignored.  This could leave a distributed transaction open.  With the SET IMPLICIT_TRANSACTION ON command, it works properly.

 

Begin Try

   Begin Transaction

      Insert myPartitionedView (col1, col2) VALUES (10, ‘a’)

   Commit Transaction

End Try

Begin Catch

   Print “error occurred”

   Rollback Transaction

End Catch

 

  1. The UI for the actual plan does not show which partitions were really touched.  You have to SET STATISTICS PROFILE ON to get the following line for each partition.

 

<RunTimeCountersPerThread Thread="0" ActualRebinds="0" ActualRewinds="0" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />

 

Debugging performance problems with Distributed Queries

The easiest method is to start SQL Profiler on each server to see which commands are being sent to which server.  In general, the startup filters will be applied and the commands will only be sent to the correct servers.  Also use the SET STATISTICS PROFILE ON that was documented above.

 

Kevin Cox, Teodoro Marinucci, Silvia Bevilacqua

(also thanks to Stefano StefaniGargi Sur and Rakesh Gujjula from the SQL Product Team)
 

posted by LifeisSimple
2011. 4. 19. 16:42 Brain Trainning/DataBase

MySpaceMySpace Uses SQL Server Service Broker to Protect Integrity of 1 Petabyte of Data

Everything about MySpace boggles the mind—from its 130 million monthly active users, to the 300,000 new users who sign up each day; and from its 8 billion friend relationships it manages, to the 34 billion e-mail messages it stores while adding 41 million more each day. The site’s 1 petabyte of data is managed by 440 Microsoft® SQL Server® instances and resides on 3PAR® Utility Storage. When MySpace needed a message queuing and delivery solution to help ensure data changes were correctly and atomically executed on all affected physical database instances, MySpace created an internal solution, called Service Dispatcher, using the Service Broker feature of SQL Server 2005. Service Broker has helped MySpace ensure data integrity across its distributed infrastructure, resulting in a better user experience. Service Broker also helps MySpace developers to roll out new services faster.

d
Situation

Since its launch in 2004, MySpace has become the world’s leading social portal for connecting people, content, and culture. MySpace empowers its global community to experience the Internet through a social lens by integrating personal profiles, photo sharing, professional and viral videos, blogs, mobile, instant messaging, and the world’s largest music community.

The numbers that frame MySpace’s success are astonishing. MySpace has more than 130 million monthly active users, with an additional 300,000 new users joining each day.

MySpace is such an attractive site that the company estimates that 40 percent of Internet users in the United States have a MySpace account, while in the United Kingdom it’s as common to have a MySpace account as it is to own a dog. What began in Santa Monica, California has grown to include an international network of more than 30 local community sites throughout North America, Latin America, Europe, Asia, and Australia.

*
* We needed to see if Service Broker could handle loads of 4,000 messages per second. Our testing found it could handle more than 18,000 messages a second. *
Christa Stelzmuller
Chief Data Architect, MySpace
*
From an IT perspective, MySpace has been in a continual race to keep up with the demands of constant growth, as MySpace is one of the fastest growing Web sites of all time. The company has long depended upon the Microsoft® Application Platform, including Microsoft SQL Server® 2005 database software. MySpace uses the Microsoft Application Platform to support:

  • 827 billion rows of data
  • 8 billion friend relationships
  • 27 billion comments
  • 34.2 billion e-mails total
  • 41 million new e-mails added per day
  • 33 million video files
  • 62,000 new videos uploaded per day

MySpace Data Services decided that the best way to handle the constant growth of its relational database stores, which now total more than 1 petabyte, was to scale horizontally and divide information across multiple instances of SQL Server. Functional separation and horizontal partitioning worked well for the company, but to help ensure data integrity while keeping up with peak loads of up to 4.4 million concurrent users, the company needed a solution that supported efficient asynchronous messaging between its 440 SQL Server instances and 1000 databases.

“Growth at MySpace has been so spectacular for so long that we initially created our horizontal scaling solution without implementing an effective messaging infrastructure linking the databases,” says Christa Stelzmuller, Chief Data Architect at MySpace. “The result was a database administrator’s nightmare, and sometimes a user experience that didn’t meet our expectations because of data integrity issues that emerged as transactions spanning multiple databases were sometimes only partially completed.”

Data integrity across all of its databases is required because each home or profile page view on MySpace is created upon demand, and because one user’s page is linked to those of friends, the dynamic page creation requires pulling data together from multiple databases. Without an efficient messaging solution to ensure transactions were atomic across all relevant databases, updates and changes entered by a user from one database might not be propagated to the tables of other databases. The result could be error messages, incomplete pages, and unhappy users.

The company needed a better solution for passing messages between databases to help ensure data integrity.

Solution

The MySpace Data Services team created a solution to act as a coordination point for message delivery across its distributed deployment of databases. The MySpace solution, called Service Dispatcher, works on a broadcast model in which the Service Dispatcher ensures that a change originating in one database is delivered to the specified target “group” relevant to the transaction.

*
* Service Broker has enabled us to reduce data errors across our distributed databases by orders of magnitude. This is significant because data errors used to be the greatest problem our group had to deal with. *
Christa Stelzmuller
Chief Data Architect, MySpace
*
At the core of Service Dispatcher is the Service Broker feature of SQL Server 2005, which provides a message-based communication platform that enables independent application components to perform as a functioning whole. Service Broker includes infrastructure for asynchronous programming that can be used for distributed applications across multiple databases.

MySpace was an early adopter of SQL Server 2005, and is in the process of upgrading its database infrastructure to SQL Server 2008.

The company uses a multi-tier architecture that includes:

  • Web Server Tier. Customers access MySpace using a browser. The Web Server tier uses Window Server® 2003 Internet Information Services (IIS) Web server technology. The Presentation tier is hosted as a Web farm with 3,000 Web servers and 800 cache servers, all running the Windows Server 2003 SP2 operating system.
  • Application Tier. MySpace coordinates the flow of data, to its users through an application tier created internally by MySpace developers using Microsoft Visual Studio® 2005, the Microsoft .NET Framework 2, and SQL CLR, which enables developers to take advantage of the common language runtime from within SQL Server. Applications include a transaction manager, service layer, pre-populator, and cache. Applications are hosted on servers running Windows Server 2003 SP2.
  • Data Tier. MySpace information—which totals more than 1 petabyte—is stored on 1,100 disks of 3PAR Utility Storage and 440 instances of SQL Server database software holding 1000 databases. The databases are being upgraded to SQL Server 2008 from SQL Server 2005 SP2. The databases are hosted on HP ProLiant DL585 computers running Windows Server 2003 Enterprise Edition SP2. Each computer has 4 dual-core AMD processors and 64 gigabytes (GB) of RAM. The data tier also includes MySpace’s internally developed Service Dispatcher application which runs on 30 computers, each running SQL Server 2005 and using the Service Broker feature. All 440 database computers run an associated Dispatcher Client application to facilitate communication with the 30 Service Dispatcher servers. The 3PAR solution, which includes 3PAR InServ® T800, S800 and S400 Storage Servers, also uses its thin snapshots to host backup copies of the SQL Server data. 3PAR is a Microsoft Gold Certified Partner.

Service Broker has enabled MySpace to perform foreign key management across its 440 database servers, activating and deactivating accounts for its millions of users, with one-touch asynchronous efficiency. MySpace also uses Service Broker administratively to distribute new stored procedures and other updates across all 440 database servers through the Service Dispatcher infrastructure.

s
x
Benefits

MySpace gained the enhanced data integrity and better user experience it had sought by using the Service Broker feature of SQL Server 2005 in creating its Service Dispatcher application to support its distributed database infrastructure. The company found Service Broker to have enterprise-class performance, and its internal developers are enjoying faster development by relying on Service Broker to handle asynchronous messaging between the database instances.

Enhanced Data Integrity 
MySpace has achieved its goal of enhancing data integrity by using the Service Broker feature of SQL Server to ensure database changes are correctly propagated across its distributed environment of some 440 database computers. Service Broker provides queuing and reliable messaging across multiple instances of SQL Server, which is exactly what MySpace needed for its Service Dispatcher solution.

“Service Broker has enabled us to reduce data errors across our distributed databases by orders of magnitude,” says Stelzmuller. “This is significant because data errors used to be the greatest problem our group had to deal with.”

Solving data integrity issues has been liberating for MySpace database administrators, who prior to the solution spent much of their time on data integrity issues.

“We estimate that our database administrators have been able to reduce the time spent on tracking down and resolving data errors by at least 80 percent,” says Stelzmuller. “This means we can spend a lot less time on operational issues and a lot more time creating solutions to carry us into the future.”

Better User Experience 
When dealing with billions of rows of data and integrating this information into the interwoven social networks and rich feature sets that are the hallmark of MySpace, data integrity is essential to support the user experience. Prior to deploying its Service Dispatcher solution powered by SQL Server Service Broker, data errors could mean missing data and frustrating error messages for users.

*
* With Service Broker we have the ability to ensure the integrity of our data, which greatly enhances the user experience. *
Christa Stelzmuller
Chief Data Architect, MySpace
*
The problems came when a change of information on one database server wasn’t propagated to all of the other servers that had relational dependencies. A user might add information to a blog that failed to show up on their profile page. “With our old system if half of the blog data was saved to the blogs database and half the data was saved to the user database, and if only one of those saves succeeded, the next time the user went back to their page they might lose their blog entry entirely or only find half of it,” Stelzmuller says. “This caused a lot of agony for our users. They would say: ‘This is the diary of my life.’”

Similar problems could occur as MySpace Data Services battled the endless tide of spammers. When removing spammers from one database, data remnants could remain on other databases, causing users to see one big red X after another. In the case of spammers, the lost names weren’t real friends. But users with no idea of who had been deleted could get anxious about what was happening with their social network.

“With Service Broker we have the ability to ensure the integrity of our data, which greatly enhances the user experience,” Stelzmuller says.

Enterprise-Class Performance 
The MySpace IT team was excited to hear about Service Broker after it was introduced as part of SQL Server 2005 because the need for an asynchronous distributed messaging solution was so essential that the team was at the time creating a solution of its own.

The initial enthusiasm about using the messaging technology already built into the database was tempered by the huge demands the team knew its infrastructure would place on any messaging solution. Before deployment, the team needed to verify that Service Broker was indeed enterprise ready. Members of the team travelled to the Microsoft SQL Server Lab in Redmond, Washington to test Service Broker performance.

“We didn’t want to start down the road of using Service Broker unless we could demonstrate that it could handle the levels of messages that we needed to support our millions of users across 440 database servers,” says Stelzmuller. “When we went to the lab we brought our own workloads to ensure the quality of the testing. We needed to see if Service Broker could handle loads of 4,000 messages per second. Our testing found it could handle more than 18,000 messages a second. We were delighted that we could build our solution using Service Broker, rather than creating a custom solution on our own.”

Faster Development 
As soon as MySpace had validated Service Broker’s enterprise-grade performance in the lab, the company’s IT group began integrating it into the Service Dispatcher solution it was already building. Service Broker helps developers compose applications from independent, self-contained components called services. Applications that require the functionality exposed in these services use messages to interact with the services, and to exchange messages between SQL Server instances. Messaging between instances was an essential element of MySpace’s distributed database infrastructure, and significantly reduced application development time.

“The ability to use Service Broker rather than creating our own messaging solution cut our development time for Service Dispatcher by half,” says Stelzmuller. “It still took some months to create our Service Dispatcher solution, but now that we have our solution up and running we’ve slashed even more development time off by offering new services that use Service Dispatcher as a base component.”

Prior to the Service Dispatcher solution, it could take several days to build and test a new service offering. “Now we can bring up a new service based on Service Broker in about 30 minutes because our developers don’t have to map out database targets or worry about complex scaling problems like conversation management,” says Stelzmuller. “They just write to our interface.”

The combination of easy programming and the ability to ensure data integrity across the distributed solution is encouraging developers to create new features and service offerings that might not have been considered earlier. “Before Service Broker and our Service Dispatcher, there was sometimes a reticence about creating new data-intensive services,” says Stelzmuller. “Now we know we can do those things, and our developers know they have a simple, quick, and uncomplicated set of tools to work with when dealing with our distributed database infrastructure.”

Summary 
In summary, MySpace found that the Service Broker feature of SQL Server 2005 provides the message queuing and asynchronous message delivery it needed to create its Service Dispatcher application to support its distributed database infrastructure of some 440 SQL Server instances holding more than 1 petabyte of information.

 

Microsoft Server Product Portfolio 
For more information about the Microsoft server product portfolio, go to:www.microsoft.com/servers/default.mspx 

Microsoft SQL Server 2005 
Microsoft SQL Server 2005 is comprehensive, integrated data management and analysis software that enables organizations to reliably manage mission-critical information and confidently run today’s increasingly complex business applications. By providing high availability, security enhancements, and embedded reporting and data analysis tools, SQL Server 2005 helps companies gain greater insight from their business information and achieve faster results for a competitive advantage. And, because it’s part of the Microsoft server product portfolio, SQL Server 2005 is designed to integrate seamlessly with your other server infrastructure investments.

For more information about SQL Server 2005, go to: www.microsoft.com/sqlserver 

For More Information 
For more information about Microsoft products and services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Information Centre at (877) 568-2495. Customers who are deaf or hard-of-hearing can reach Microsoft text telephone (TTY/TDD) services at (800) 892-5234 in the United States or (905) 568-9641 in Canada. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information using the World Wide Web, go to: www.microsoft.com 

For more information about 3PAR products and services, call (510) 413-5999 or visit the Web site at:WWW.3PAR.COM 

For more information about MySpace products and services, visit the Web site at: www.myspace.com 

 

This case study is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.

Document published June 2009

posted by LifeisSimple
2011. 3. 25. 16:51 Brain Trainning/Storage
출처 : http://social.technet.microsoft.com/wiki/contents/articles/sqlio-disk-subsystem-benchmark-tool-troubleshooting-hangs-and-errors.aspx

SQLIO Disk Subsystem Benchmark Tool: Troubleshooting Hangs and Errors

The SQLIO Disk Subsystem Benchmark Tool is a tool provided by Microsoft (SQLIO from Microsoft Download), which can be used to determine the I/O capacity of a given configuration. However, it comes with the disclaimer that SQLIO is provided ‘as is’ and there is no support offered for any problems encountered when using the tool. Please refer to the EULA.doc for the license agreement prior to using this tool.

Tips on how to prevent or resolve issues when running the SQLIO tool, especially on Windows 7 and Windows Server 2008 R2:

  1. Ensure that you have created an exception for SQLIO in Data Execution Prevention (DEP). In Windows 7 or Windows Server 2008 R2, you can create exceptions in DEP by opening System Properties (sysdm.cpl) and then clicking Advanced. Then, in Performance, click Settings. Click Data Execution Prevention. IfTurn on DEP for all programs and services except those I select is selected, click Add, then type the full path to SQLIO.EXE, including SQLIO.EXE. For example, C:\Program Files (x86)\SQLIO\SQLIO.EXE. If the other option, Turn on DEP for essential Windows programs and services only is selected, you do not need create an exception .

  • If your SQLIO commands contain -fsequential and SQLIO is crashing, ensure you define a params.txt file that sets the test file size before testing.
  • Ensure that all programs that are operating or could operate on the target drive for testing are off or suspended during the test. Two common examples include:
    1. Ensure that you configure exceptions in any virus scanning software that you have running or that could run during your test for both SQLIO.EXE and the target volume of the test.
    2. Ensure that you disable any drive indexing software. 

 To learn more about SQLIO, see the following related references:

posted by LifeisSimple
prev 1 2 3 4 5 6 7 8 9 next